Building FastDFS file system in Docker (multi-image tutorial)

About FastDFS

FastDFS is an open source lightweight distributed file system developed in C language. It manages files. Its main functions include: file storage, file synchronization, file access (file upload/download), etc. It is particularly suitable for online services based on files, such as picture websites, video websites, etc.

Side note: FastDFS is a personal project of Yu Qing from Alibaba. Since the development of FastDFS open source in 2008, it has been highly respected in some Internet startups. The GitHub open source address is: https://github.com/happyfish100/fastdfs

This article is based on Docker. The following is the main content:

1. Search for images

Execute the command:

docker search fastdfs

Then the interface will search out a lot of images, which one should I choose?

As usual, we should choose the one with the largest number of starts. What? Are you worried about the start number and afraid of being hacked? Ok, let's go to the image repository and take a look:

Supplement: In fact, we can see some differences by looking at the descriptions of these images. For example, ygqygq2/fastdfs-nginx is an image that integrates fastdfs with Nginx. So a new question arises: why integrate Nginx? Because for fastdfs, after installation, it can only be accessed on the local machine... I upload files for access, so in order to provide web access, then Nginx needs to be integrated, but if you want to do Nginx by yourself... then you definitely won't choose it. The season/fastdfs image is relatively pure. This article is based on the season/fastdfs image, and the Nginx configuration is also included later...

2. Install the image

Execute the command:

docker pull season/fastdfs:1.2

The reason why we choose version 1.2 is to prevent this article from being unsuitable for the latest version after another update. The execution screenshots are as follows:

3.

Create a container and mount the directory

Before creating a container, let's briefly talk about FastDFS. The FastDFS system has three roles:

• 跟蹤服務器: Tracker Server is mainly responsible for scheduling and balancing. It is responsible for managing all storage servers and groups. After startup, each storage will connect to Tracker to inform the group it belongs to and maintain periodic heartbeats.
• 存儲服務器Server: Storage servers mainly provide capacity and backup services. They are organized into groups, and each group can have multiple storage servers, with data backed up by each other.
• 客戶端: The server that uploads and downloads data, that is, the server where our own project is deployed.

Speaking of this, I want to tell my friends that next we may create three containers: tracking server container, storage server container, and client container...

3.1. Create the required directories

We first create some necessary directories (data directory, data storage directory, etc.), and execute the command:

mkdir -p /usr/local/server/fastdfs/tracker/data
mkdir -p /usr/local/server/fastdfs/storage/data
mkdir -p /usr/local/server/fastdfs/storage/path

3.2. Create a tracker container (tracking server container)

Execute the command:

docker run -id --name tracker \
-p 22122:22122 \
--restart=always --net host \
-v /usr/local/server/fastdfs/tracker/data:/fastdfs/tracker/data \
season/fastdfs:1.2 tracker

Let me repeat the above command:

• -d: Run the container in the background
• -p: specifies the network port used inside the container to be mapped to the host we use
• --name: Specifies the name of the container to be created
• -v: Mount directory between container and host

3.3. Create a storage container (storage server container)

Execution command ( not the final execution command, please change it to your own IP address):

docker run -id --name storage \
--restart=always --net host \
-v /usr/local/server/fastdfs/data/storage:/fastdfs/store_path \
-e TRACKER_SERVER="10.211.55.4:22122" \
season/fastdfs:1.2 storage

3.4、Client test

After the two containers are created, but they are not actually associated at this time, we enter the tracker container and test it through client.conf:

docker exec -it tracker bash
cd /etc/fdfs/
ls
cat client.conf

The output client.conf is all default configuration, we can find the track_server address:

Try to connect:

fdfs_monitor client.conf

Then you will get something like this:

ERROR - file: ../common/connection_pool.c, line: 84, connect to 192.168.209.121:22122 fail, errno: 110, error info: Connection timed out

That is, before modifying the client configuration, the default tracker connects to the 192.168.209.121 address, so the next step is to configure the client configuration file.

3.4. Modify the client.conf configuration file

After entering the tracker container through docker exec -it tracker bash you cannot use the vi vim command, so simply copy the client.conf configuration file, modify it on the host machine, and then paste it back.

Execute the command:

docker cp trakcer:/etc/fdfs/client.conf /usr/local/server/fastdfs/

We copy the configuration file to the /usr/local/server/fastdfs/ directory through the docker command.

We edit this file and modify the tracker_url address ( vi client.conf ):

Save the changes after modification, and then replace the modified file back. Instruction command:

docker cp /usr/local/server/fastdfs/client.conf tracker:/etc/fdfs

At this point, the configuration file has been modified. Next, we create a file upload test.

4. File upload test

Execute the command to enter the tracker container:

docker exec -it tracker bash

Just create a txt file:

echo "niceyoo" > niceyoo.txt

Then upload the niceyoo.txt file to the server using the fdfs_upload_file command:

fdfs_upload_file /etc/fdfs/client.conf niceyoo.txt

If the following error is reported at this time:

Then create this path and skip if there is none:

mkdir -p /home/yuqing/fastdfs

Create the directory and try submitting again:

We copy the file name: group1/M00/00/00/CtM3BF84iz2AWE_JAAAACBfWGpM793.txt . This name will be used later as the access file path when configuring nginx.

Since we mounted the host directory before, let's go to cd /usr/local/server/fastdfs to take a look:

cd /usr/local/server/fastdfs/data/storage/data
ls

The output is as follows:

5. Configure Nginx

As mentioned in the previous supplement, by default the uploaded files can only be accessed locally. Of course, this is definitely not acceptable, so we need to configure Nginx to help us achieve the effect of Web access.

Create the nginx directory:

mkdir -p /usr/local/server/fastdfs/nginx/

Copy the nginx configuration file in the storage container:

docker cp storage:/etc/nginx/conf/nginx.conf /usr/local/server/fastdfs/nginx/

Modify the configuration in nginx:

vi /usr/local/server/fastdfs/nginx/nginx.conf

Find the local node and modify it to:

 location / {
    root /fastdfs/store_path/data;
    ngx_fastdfs_module;
 }

The relevant screenshots are as follows:

The next step is to create the nginx container and point it to the configuration file just created.

Friends who have not installed nginx can also execute the following instructions. By default, it will help you pull the image and install it.

Note: Please modify the IP address before executing

docker run -id --name fastdfs_nginx \
--restart=always \
-v /usr/local/server/fastdfs/data/storage:/fastdfs/store_path \
-v /usr/local/server/fastdfs/nginx/nginx.conf:/etc/nginx/conf/nginx.conf \
-p 8888:80 \
-e TRACKER_SERVER=10.211.55.4:22122 \
season/fastdfs:1.2 nginx

At this point, let's take a look at the containers that have been created and started:

Let's test the previously uploaded file group1/M00/00/00/CtM3BF84iz2AWE_JAAAACBfWGpM793.txt and execute the following command:

curl -i http://127.0.0.1:8888/group1/M00/00/00/CtM3BF84iz2AWE_JAAAACBfWGpM793.txt

Execution Result:

At this point, the whole process has been successfully built. Let's access it through the browser:

How to use it in the project after the construction is completed?

SpringBoot integrates FastDFS dependency to implement file upload

This is the end of this article about building FastDFS file system in Docker (multi-picture tutorial). For more relevant content about building FastDFS in Docker, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!