Implementation steps for building FastDFS file server in Linux

This article mainly introduces how to build a FastDFS file server on a Linux server. It is roughly divided into 9 steps, as the content is relatively complicated. Let me take you into the pit below!

First, let me briefly introduce FastDFS, which is a distributed file system written in C language and led by Mr. Yu Qing, a senior architect of Taobao, to open source. Suitable for small and medium-sized enterprises, files are not divided into blocks.

There are mainly Tracker (management) and Storage (storage).

Return path = group name/virtual drive letter {M00/00/02}/file name

1. Software Package

The following is the installation package for FastDFS

Step 1: Enter https://sourceforge.net/projects/fastdfs/files in the browser address bar, and you will see the interface shown in the following figure.

Step 2: Download the FastDFS Server installation package. Click "FastDFS Server Source Code" in the above picture and you will see the interface shown below. Select the 2014-12-02 version because it is the most stable version.

Step 3: We will see the interface as shown below, we click "FastDFS_v5.05.tar.gz" to download.

The following is the downloaded

Or download it yourself

wget https://github.com/happyfish100/libfastcommon/archive/V1.0.7.tar.gz
wget http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Nginx%20Module%20Source%20Code/fastdfs-nginx-module_v1.16.tar.gz
wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz
wget http://nginx.org/download/nginx-1.12.1.tar.gz

2. Install gcc

Check whether GCC is installed on the system

gcc –version

Install GCC

yum -y install gcc gcc-c++ autoconf pcre pcre-devel make automake
yum -y install wget httpd-tools vim

Just use gcc as a C language compiler and g++ as a C++ language compiler. (Zhihu)

3. Install libfastcommon

Enter the libfastcommon compressed package directory to decompress, compile, and install

tar -zxvf libfastcommonV1.0.7.tar.gz

Enter the unzipped libfastcommon-1.0.7 directory and compile

./make.sh

Install

./make.sh install

Finally, the libfastcommon.so file is in usr/lib. Please modify the copied path according to your actual file path.

cp /data/fastDFS/software/libfastcommon-1.0.7/src/libfastcommon.so /usr/lib

4. Install FastDFS

Unzip FastDFS_v5.05.tar.gz , then compile and install FastDFS

tar -zxvf FastDFS_v5.05.tar.gz
./make.sh
./make.sh install

After successful installation, copy the files in the directory conf to the /etc/fdfs directory:

cp /data/fastDFS/software/FastDFS/conf/* /etc/fdfs/

5. Install tracker

Enter the /etc/fdfs directory and modify the tracker.conf file. If it does not exist, copy the tracker.conf.sample file to tracker.conf and then modify it:

base_path=/home/yuqing/fastdfs >>> base_path=/home/logs/fastdfs (Note: log file)

http.server_port=8080 >>> http.server_port=80 (Note: port 80 is the default for convenience)

store_group=group1 (Note: group name)

Among them, /home/

logs

If the /fastdfs directory does not exist, create one.

After that, start tracker and check whether it is started successfully (the following prompt appears, indicating that the startup is successful): restart

fdfs_trackerd /etc/fdfs/tracker.conf start
netstat -unltp | grep tracker 

6. Install storage

Since both storage and tracker run fastDFS programs, each server deploys a fasfDFS, one server is tracker, and the other is storage.

Modify the storage.conf file. If it does not exist, copy the storage.conf.sample file to storage.conf , and then modify it:

base_path=/home/yuqing/fastdfs >>> base_path=/home/logs/storage (log path)

store_path0=/home/yuqing/fastdfs >>> store_path0=/home/data/storage (actual storage file path, multiple paths can be configured)

tracker_server=192.168.209.121:22122 >>> tracker_server=192.168.2.231:22122 (connect to the tracker server address)

group_name=group1 (must be the same as the tracker group name) http.server_port=80 (this port must also be changed)

If /home/logs/storage and /home/data/storage do not exist, create the directory

Then, start storage and check whether it is successful (the following prompt appears, indicating that the startup is successful):

fdfs_storaged /etc/fdfs/storage.conf start
netstat -unltp | grep storage 

Finally, check whether tracker and storage are communicating:

fdfs_monitor /etc/fdfs/storage.conf

As shown below, ACTIVE appears, indicating that both are started normally. Now you can test uploading files.

7. Test image upload

Tracker and storage have been installed. Use the command to test file upload:
FastDFS provides a file upload command: usr/bin/fdfs_test to test file upload. To test upload, you need to connect to the tracker server and the storage server. Therefore, you need to specify a configuration file: client.conf configuration file, and connect to the tracker server through Client.conf.

Modify client.conf under /etc/fdfs

base_path=/home/logs/client (log directory)
tracker_server=192.168.2.231:22122 (tracker port)

Test Command

/usr/bin/fdfs_test /etc/fdfs/client.conf upload anti-steal.jpg

If the return address is as shown below, it is successful

8. Install Nginx.

Install other environments required by Nginx. GCC has been installed above. I didn't take this step when installing

#gcc installation yum install gcc-c++

#PCRE pcre-devel install yum install -y pcre pcre-devel

# Install zlib yum install -y zlib zlib-devel

#OpenSSL installation yum install -y openssl openssl-devel

Unzip, compile, and install.

tar -zxvf nginx-1.12.1.tar.gz
cd nginx-1.12.1
./configure
make
make install

9. Install fastdfs-nginx-module

First, unzip fastdfs-nginx-module_v1.16.tar.gz and modify the /fastdfs-nginx-module/src/config file. Remove all local (three)

Copy the library file libfdfsclient.so in the usr/lib64 directory

cp /usr/lib64/libfdfsclient.so /usr/lib

Execute the following command in the nginx installation directory: Add the module to nginx. Add modules by setting installation parameters.

cd nginx-1.12.1
./configure --add-module=../fastdfs-nginx-module/src

Note: I have nginx-1.12.1 and fastdfs-nginx-module in the same directory, so write it like this

Reinstall the compiler

make && make install

View Nginx modules

/usr/local/nginx/sbin/nginx -V

Copy the configuration file in the fastdfs-nginx-module source code to the /etc/fdfs directory and modify

cd fastdfs-nginx-module/src
cp mod_fastdfs.conf /etc/fdfs/

Enter /etc/fdfs/ and modify mod_fastdfs.conf as follows. Other settings are by default.

#Connection timeout connect_timeout=10

Tracker Server
tracker_server=192.168.2.231:22122

# StorageServer default port storage_server_port=23000

# If the uri of the file ID contains /group**, set it to true
url_have_group_name = true

# The store_path0 path of the Storage configuration must be consistent with the one in storage.conf store_path0=/home/data/storage

# the base path to store log files
base_path=/home/logs/storage

Configure iNginx, enter the /usr/local/nginx/conf directory and modify nginx.conf

Note: No matter where you unzip and install. You must go to this directory /usr/local/nginx/conf and modify nginx.conf

vim nginx.conf

Modify the configuration, other defaults

Add the fastdfs-nginx module to port 80

location ~/group([0-9])/M00 {
    ngx_fastdfs_module;
}

Note that the following #user nobody; is changed to #user root;

#user nobody;
worker_processes 1;

#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;

#pid logs/nginx.pid;


events {
    worker_connections 1024;
}


http {
    include mime.types;
    default_type application/octet-stream;

    #log_format main '$remote_addr - $remote_user [$time_local] "$request" '
    # '$status $body_bytes_sent "$http_referer" '
    # '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log logs/access.log main;

    sendfile on;
    #tcp_nopush on;

    #keepalive_timeout 0;
    keepalive_timeout 65;

    #gzip on;

    server {
        listen 80;
        server_name 192.168.2.231;
        location ~/group([0-9])/M00 {
            ngx_fastdfs_module;
        }
        #charset koi8-r;

        #access_log logs/host.access.log main;

        location / {
            root html;
            index index.html index.htm;
        }

        #error_page 404 /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
            root html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        # proxy_pass http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #root html;
        # fastcgi_pass 127.0.0.1:9000;
        # fastcgi_index index.php;
        # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
        #include fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        # deny all;
        #}
    }


    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    # listen 8000;
    # listen somename:8080;
    # server_name somename alias another.alias;

    # location / {
    #root html;
    # index index.html index.htm;
    # }
    #}


    # HTTPS server
    #
    #server {
    # listen 443 ssl;
    # server_name localhost;

    # ssl_certificate cert.pem;
    # ssl_certificate_key cert.key;

    # ssl_session_cache shared:SSL:1m;
    #ssl_session_timeout 5m;

    # ssl_ciphers HIGH:!aNULL:!MD5;
    # ssl_prefer_server_ciphers on;

    # location / {
    #root html;
    # index index.html index.htm;
    # }
    #}
}

Notice:

The listen 80 port value should correspond to http.server_port=80 in /etc/fdfs/storage.conf (changed to 80 earlier). If you change to another port, you need to unify it and open the port in the firewall.

For location configuration, if there are multiple groups, configure location ~/group([0-9])/M00. If there are no groups, no group is required.

Create a soft link in the file storage directory and link it to the directory where the data is actually stored. Note that the multiple data directories behind this file storage location are automatically generated by the system.

ln -s /home/data/storage/data /home/data/storage/data/M00

Start nginx

#Start nginx
cd /usr/local/nginx/sbin/
./nginx directly starts /usr/local/nginx/sbin/nginx

#Set up vim startup /etc/rc.local
/usr/local/nginx/sbin/nginx

# Set execution permissions chmod 755 rc.local

Check whether Nginx is started

ps -ef | grep nginx

OK, remember that the security group and firewall settings of the cloud service need to be 80, 22122, and 23000. If you don't know how to do this, please refer to the firewall settings in Tencent Cloud Server Tomcat Port Unreachable

Access successfully in the address bar http://192.175.231.128/group1/M00/00/00/rBAACVzBeU2AQBKJAAOHDqS1H9o350.jpg

Report an error! Does not exist. There are problems 110, 120, 119

Notice:

1. The FastDFS service port of Storage2 and Storage3 in the same group of group2 must be consistent: port=23000.

2. A server can have multiple groups but cannot have multiple storages in the same group. The log will report an error. The reason for the error is "Note 1"

3. Before Version 4.05, fastdfs internally bound libevent as the http server. Versions after Version 4.05 deleted the built-in web http service. The built-in web http service is a burden and it is better not to use it!

4. When starting the storage server, it is always in a deadlock state. When starting the storage server, the storage will connect to the tracker server. If it cannot connect, it will keep retrying. The startup is not complete until the connection is successful! If there are 2 tracker servers in the cluster,

If one of the trackers is not started, the storage server may remain in a deadlock state.

This is the end of this article about the implementation steps of building a FastDFS file server in Linux. For more relevant content about building a FastDFS file server in Linux, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!