Implementation of fastdfs+nginx cluster construction

1. Introduction to fastdfs

1. What is fastdfs

Fastdfs is a lightweight open source distributed file system;
Fastdfs mainly solves the problem of large-capacity file storage and high-concurrency access, and achieves load balancing when accessing files;
Fastdfs implements software RAID, which can use cheap IDE hard disks for storage and support online expansion of storage servers. It supports saving only one copy of files with the same content, thus saving disk space.
Fastdfs can only be accessed through the Client API and does not support POSIX access methods;
Fastdfs is particularly suitable for large and medium-sized websites to store resource files (such as pictures, documents, audio, video, etc.).

2.fastdfs framework diagram

The author (YuQing) gives the framework diagram as follows:

in:

Tracker Server: Tracking server mainly performs scheduling work and plays a role in load balancing in access. Records the status of the storage server and is the hub connecting the Client and the Storage server.
Storage Server : Storage server, files and meta data are stored on the storage server
group: Group, also called volume. Files on servers in the same group have exactly the same file identifier: It consists of two parts: group name and file name (including path)
meta data: file-related attributes, in key value pair format, such as width=1024,height=768

3. Upload file process

(1) The client asks the tracker for the storage to upload to, without any additional parameters;
(2) The tracker returns an available storage;
(3) The client communicates directly with the storage to complete the file upload.

4. Download file process

(1) The client asks the tracker for the storage of the downloaded file, and the parameters are the file identifier (group name and file name);
(2) The tracker returns an available storage;
(3) The client communicates directly with the storage to complete the file download.

5. Synchronization mechanism

(1) Storage servers in the same group are peers, and file upload, deletion and other operations can be performed on any storage server;
(2) File synchronization is only performed between storage servers in the same group, using the push method, that is, the source server synchronizes to the target server;
(3) Only the source data needs to be synchronized, and the backup data does not need to be synchronized again, otherwise a loop will be formed;
(4) There is an exception to the second rule above. When a new storage server is added, an existing storage server will synchronize all existing data (including source data and backup data) to the new server.

6. Tracker server directory structure

(1) data

storage_groups.dat: storage group information

storage_servers.dat: storage server list

(2) logs

trackerd.log: tracker server log file

7. Storage server directory structure

1) data

.data_init_flag: current storage server initialization information

storage_stat.dat: Current storage server statistics

sync: stores files related to data synchronization

binlog.index: current binlog file index number

binlog.###: stores update operation records (logs)

${ip_addr}_${port}.mark: stores the completion status of synchronization

(2) logs

storaged.log: storage server log file

The above theoretical reference:

ChinaUnix Forum: http://bbs.chinaunix.net/forum-240-1.html

GitHub download address: https://github.com/happyfish100

2. fastdfs installation and testing

1. Download

Need to download: libfastcommon fastdfs (versions before 2.0 need to rely on libevent)

2. Installation

(1) Install libfastcommon

$ tar xzvf libfastcommon-1.0.38.tar.gz
$ cd libfastcommon-1.0.38
$ ./make.sh
$ sudo ./make.sh install

(2) Install fastdfs

$ tar xzvf fastdfs-5.11.tar.gz
$ cd fastdfs-5.11
$ ./make.sh
$ sudo ./make.sh install

3. Configuration

Cluster building topology diagram (due to insufficient hard disk, 192.168.31.4 and 192.168.31.14 are not available now, and will be added after capacity expansion)

(1) Configure tracker (192.168.31.95/192.168.31.99)

$cd /etc/fdfs/

The directory files are as follows:

Then, execute as follows:

$ sudo su
#cp tracker.conf.sample tracker.conf

Configure the tracker.conf file:

vim tracker.conf

Modify: base_path=/home/fastdfs/tracker (Note: This directory must exist before starting, otherwise an error will be reported and the path cannot be found, which means that a new tracker directory needs to be created manually)

start up:

/etc/init.d/fdfs_trackerd start

(2) Configure storage (192.168.31.2/192.168.31.12)

$ cd /etc/fdfs
$ sudo su
#cp storage.conf.sample storage.conf

Revise:

group_name = group1 (192.168.31.2) or group2 (192.168.31.12)

base_path=/home/fastdfs/storage

store_path0=/home/fastdfs/storage

Modification: tracker service ip and port:

tracker_server=192.168.31.95:22122 (Note: here 31.2 tracker_server configures 31.95, 31.12 tracker_server configures 31.99)

start up:

/etc/init.d/fdfs_storaged start

View the corresponding configuration information of fastdfs

# fdfs_monitor /etc/fdfs/storage.conf

Success is considered to be successful if the following information is present:

Test upload, select 192.168.31.99 tracker machine as fdfs_client

$ cd /etc/fdfs

$ sudo su

# cp client.conf.sample client.conf

# vim client.conf

Revise:

base_path = /home/fastdfs

tracker_server=192.168.31.99:22122

Test uploading the net.png file:

fdfs_upload_file /etc/fdfs/client.conf net.png

3. nginx configuration

Need to install:

fastdfs-nginx-module-1.20 Something similar to a plugin

nginx-1.14.2.tar.gz

To install nginx, you need to install:

openssl zlib pcre

Insert astdfs-nginx-module into nginx (need to configure and make&&make install nginx)

./configure --add-module=../../fastdfs-nginx-module/src/

An error will be reported: The fastdfs-nginx-module-1.20/src/config file needs to be modified

ngx_addon_name=ngx_http_fastdfs_module

if test -n "${ngx_module_link}"; then
  ngx_module_type=HTTP
  ngx_module_name=$ngx_addon_name
  ngx_module_incs="/usr/local/include"
  ngx_module_libs="-lfastcommon -lfdfsclient"
  ngx_module_srcs="$ngx_addon_dir/ngx_http_fastdfs_module.c"
  ngx_module_deps=
  CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
  .auto/module
else
  HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
  NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
  CORE_INCS="$CORE_INCS /usr/local/include"
  CORE_LIBS="$CORE_LIBS -lfastcommon -lfdfsclient"
  CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
fi

Modified to:

ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"

ngx_addon_name=ngx_http_fastdfs_module

if test -n "${ngx_module_link}"; then
  ngx_module_type=HTTP
  ngx_module_name=$ngx_addon_name
  ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
  ngx_module_libs="-lfastcommon -lfdfsclient"
  ngx_module_srcs="$ngx_addon_dir/ngx_http_fastdfs_module.c"
  ngx_module_deps=
  CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
  .auto/module
else
  HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
  NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
  CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
  CORE_LIBS="$CORE_LIBS -lfastcommon -lfdfsclient"
  CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
fi

Then reconfigure and make && make install

Next, cp mod_fastdfs.conf /etc/fdfs/ and modify the mod_fastdfs.conf file

$ cd fastdfs-nginx-module-1.20/src
$ cp mod_fastdfs.conf /etc/fdfs/

Modify mod_fastdfs.conf

tracker_server=192.168.31.99:22122
store_path0=/home/fastdfs/storage
group_name=group1

[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/home/fastdfs/storage


[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/home/fastdfs/storage

fastdfs-5.11/conf/

$ cp http.conf /etc/fdfs/
$ cp mime.types /etc/fdfs/

Modify: http.conf

http.anti_steal.token_check_fail=/home/fastdfs/fastdfs-5.11/conf/

Modify nginx.conf

cd /usr/local/nginx/sbin
vim ../conf/nginx.conf

The port number 8888 corresponds to http.server_port=8888 in /etc/fdfs/storage.conf

Upload a picture

http://192.168.31.12:8888/group1/M00/00/00/wKgfDFzxVTyAM4suABWWvfB6x_k962.png

This is the end of this article about the implementation of fastdfs+nginx cluster construction. For more related fastdfs+nginx cluster construction content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in: