Use Docker to build a Redis master-slave replication cluster

In a cluster with master-slave replication mode, there is usually one master node and two or more slave nodes. Data written to the master node will be copied to the slave node. In this way, once the master node fails, the application system can switch to the slave node to read and write data, which can improve the availability of the system. Moreover, if the default read-write separation mechanism in the master-slave replication mode is adopted, the system's cache read-write performance can be further improved. Therefore, for systems with low performance and real-time requirements, the master-slave replication mode is sufficient to meet general performance and security requirements.

1 Overview of Master-Slave Replication Mode

In actual applications, if there are corresponding settings, after writing data to a Redis server, the data can be copied to another (or multiple) Redis servers. Here, the data source server is called the master server, and the server where the copied data is located is called the slave server.

This master-slave replication mode can bring two benefits. First, write operations can be concentrated on the master server and read operations can be concentrated on the slave server, which can improve read and write performance. Second, due to data backup, data security can be improved. For example, when the master Redis server fails, it can quickly switch to reading data from the slave server.

If the concurrency requirement is not high in the project, or even if the data cannot be read from the Redis cache, it will not cause much damage to the performance, then a master-slave replication mode can be used. The effect diagram is shown in the figure below.

You can also set up a one-master-multiple-slave replication effect. The following figure shows the corresponding effect diagram, that is, the data written to the master node will be synchronized to the two slave nodes. Other one-master-multiple-slave modes are very similar to this.

Regarding the master-slave replication mode, please pay attention to the following points.

First, a master server can have one or more slave servers, and even a slave server can have another slave server, but when copying data, the data of the master server can only be copied to the slave server, not vice versa.

Second, a slave server can only follow one master server, and a one-slave-multiple-master mode is not possible.

Third, in versions after Redis 2.8, an asynchronous replication mode is used, that is, when performing master-slave replication, it will not affect the read and write data operations on the master server.

2 Use commands to build a master-slave cluster

Here we will use Docker containers to build a cluster with one master and two slaves. When configuring the master-slave relationship, you need to use the slaveof command on the slave node. The specific steps are as follows.

The first step is to open a command window and run the following command to create a Redis container named redis-master. Note that its port is 6379.

docker run -itd --name redis-master -p 6379:6379 redis:latest

In the second step, open a new command window and run the following command to create a container named redis-slave1. Note that its port is 6380. Please note that this is running on one computer, so the port number is used to distinguish the master Redis container from the other two slave Redis containers. If in a real project, multiple Redis servers will be deployed on different servers, so port 6379 can be used for all of them.

docker run -itd --name redis-slave1 -p 6380:6380 redis:latest

The third step is to return to the command window containing the redis-master container and run the docker inspect redis-master command in it to view the information of the redis-master container. You can see the IP address of the container through the IPAddress item, which is 172.17.0.2 here. If in a real project, the IP address of the Redis server is fixed, and the IP address of the Redis server started by the Docker container is dynamic, so the above command is used here to obtain the IP address.

Step 4. In the command window of the redis-master container, run the docker exec -it redis-master /bin/bash command to enter the command line window. Use the redis-cli command to enter the Redis client command line, and then use the info replication command to view the status of the current master-slave mode. You can see some results as shown below.

 c:\work>docker exec -it redis-master /bin/bash
 root@9433cd584d80:/data# redis-cli
 127.0.0.1:6379> info replication
 # Replication
 role:master
 connected_slaves:0

From the output of line 5, we can see that the role of the current reids-master container in the master-slave mode is the "master server". From the output of line 6, we can see that the master server currently has no slave server.

Similarly, in the command window of the redis-slave1 container, use the docker exec -it redis-slave1 /bin/bash command to enter the command line window of the container, use the redis-cli command to enter the client command line, and use the info replication command to view the master-slave mode status of the Redis server. Some of the results are shown below.

 c:\work>docker exec -it redis-slave1 /bin/bash
 root@2e3237c60211:/data# redis-cli
 127.0.0.1:6379> info replication
 # Replication
 role:master
 connected_slaves:0

Since the master-slave mode has not been set through the command line at this time, the output results of lines 5 and 6 still show that the current server is the "master server" and there is no slave server.

Step 5. In the command window of the redis-slave1 container, run the following slaveof command to specify the current Redis server as the slave server. The format of this command is slaveof IP address port number, which refers to the master server at 172.17.0.2:6379.

slaveof 172.17.0.2 6379

After running the command, run info replication again in the redis-slave1 client and you will see some results as shown below. From the result in line 3, we can see that the redis-slave1 server has become a slave server, and from the output in lines 4 and 5, we can confirm that the slave server is subordinate to the Redis master server at 172.17.0.2:6379.

127.0.0.1:6379> info replication
 # Replication
 role:slave
 master_host:172.17.0.2
 master_port:6379

At this time, return to the command window of the redis-master container and run the info replication command again in the Redis client to view the master-slave status. You can see some results as shown below. From the output of line 4, we can see that the Redis master server already has a slave server.

 127.0.0.1:6379> info replication
 # Replication
 role:master
 connected_slaves:1

Step 6. Open a new command window and run the following command in it to start a new Redis container named redis-slave2. Note that its port is 6381.

docker run -itd --name redis-slave2 -p 6381:6381 redis:latest

Then run the docker exec -it redis-slave2 /bin/bash command to enter the command line window of the container, then use the redis-cli command to enter the client, run the slaveof 172.17.0.2 6379 command, set this Redis server as a slave server, and connect it to the master Redis server where the redis-master container is located.

After the connection is completed, return to the command line window where the redis-master container is located and run the info replication command. You can see the following partial output. From the output of line 4, you can see that the master server is currently connected to two slave servers.

127.0.0.1:6379> info replication
 # Replication
 role:master
 connected_slaves:2

So far, the master-slave mode of one master and two slaves has been configured. At this time, if you run the get name command in the two slave servers, the return value is empty. If you go to the command line window where the redis-master container is located, run set name Peter in it, and then run the get name command in the two slave servers, you can see the return value. This means that the master-slave mode is configured successfully, and the data in the master server will be automatically synchronized to each slave server.

3. Build a master-slave cluster through configuration

In the project, you can use the slaveof command to build a master-slave cluster, or you can use configuration parameters to build it. The specific steps are as follows.

The first step is to build the command for the main server redis-master. The following command is still used, and port 6379 is still used here.

docker run -itd --name redis-master -p 6379:6379 redis:latest

Use the docker inspect redis-master command to confirm that the IP address of the container where the Redis server is located is still 172.17.0.2.

The second step is to go to the C:\work\redis\redisConf directory, create a configuration file redisSlave1.conf, and write the following content in it.

port 6380

slaveof 172.17.0.2 6379

Through the command in line 1, set the Redis port to 6380. Through the slaveof configuration in line 2, set the Redis server to "slave mode" and connect to the master server where redis-master is located.

The third step is to run the following command in a new command window to create a Redis server named redids-slave1. The working port of the server is 6380, and the parameter after redis-server specifies to load the redisSlave1.conf configuration file when starting the Redis server.

docker run -itd --name redis-slave1 -v C:\work\redis\redisConf:/redisConfig:rw -p 6380:6380 redis:latest redis-server /redisConfig/redisSlave1.conf

Then use the docker exec -it redis-slave1 /bin/bash command to enter the command line of the container. Since the Redis working port here has become 6380, you need to use the redis-cli -h 127.0.0.1 -p 6380 command to enter the Redis client. If you run the info replication command in it, you can see the following partial results, which can further confirm that the redis-slave1 server is subordinate to the redis-master server.

 root@80e7ae14a322:/data# redis-cli -h 127.0.0.1 -p 6380
 127.0.0.1:6380> info replication
 # Replication
 role:slave
 master_host:172.17.0.2
 master_port:6379

Step 4. Go to the C:\work\redis\redisConf directory, create a configuration file redisSlave2.conf, and write the following content in it.

port 6381

slaveof 172.17.0.2 6379

Port 6381 is used here, and the slaveof command is also used to connect to the redis-master server. Then run the following command in a new command window to create a Redis server named redids-slave2. The working port of the server is 6381, and the parameters after redis-server are used to specify that the redisSlave2.conf configuration file is loaded when the Redis server is started.

docker run -itd --name redis-slave2 -v C:\work\redis\redisConf:/redisConfig:rw -p 6381:6381 redis:latest redis-server /redisConfig/redisSlave2.conf

Then use the docker exec -it redis-slave2 /bin/bash command to enter the command line of the container. Since the Redis working port has become 6381, you need to use the redis-cli -h 127.0.0.1 -p 6381 command to enter the Redis client. Here you can use the info replication command to confirm the configuration effect. Some of the running results are shown below.

root@6017108b97c4:/data# redis-cli -h 127.0.0.1 -p 6381
 127.0.0.1:6381> info replication
 # Replication
 role:slave
 master_host:172.17.0.2
 master_port:6379

So far, the configuration of the master-slave replication cluster using the configuration file has been completed. At this time, if you run the set age 18 command in the client where the master server redis-master is located, and then run the get age command in the two slave servers redis-slave1 and redis-slave2, you can see the value of age, which can once again confirm that the data can be synchronized between the master and slave servers.

4 Configuring the read-write separation effect

If you run the info replication command on the two slave servers redis-slave1 and redis-slave2 configured above, you can also see the configuration "slave_read_only:1", which means that the slave server is "read-only" by default. If you enter set val 1 in the Redis client command line of redis-slave1, you will see the error shown in the second line below, which can further verify the "read-only" attribute of the Redis server.

127.0.0.1:6380> set val 1

(error) READONLY You can't write against a read only replica.

For the Redis slave server, it is recommended to use the default "read-only" configuration, because in the project, data is generally not written to the "slave server" as the data synchronization destination. If it is really necessary for business, you can set the "readable and writable" effect by following the steps below.

The first step is to add a line of "slave-read-only no" configuration to the redisSlave2.conf configuration file mentioned above to specify that the server is readable and writable.

In the second step, if the redis-slave2 container mentioned above is still active, you need to stop the container with docker stop redis-slave2, and then delete the container with the docker rm redis-slave2 command. Then you can create the redis-slave2 container again with the following command.

docker run -itd --name redis-slave2 -v C:\work\redis\redisConf:/redisConfig:rw -p 6381:6381 redis:latest redis-server /redisConfig/redisSlave2.conf

In the redisSlave2.conf configuration file following the redis-server command, the "slave-read-only no" configuration item has been used to set the "readable and writable" mode.

In the third step, use the docker exec -it redis-slave2 /bin/bash command to enter the command line of the container, and then use the redis-cli -h 127.0.0.1 -p 6381 command to enter the Redis client. At this time, if you run the set val 1 command again, you can successfully write data.

5. Use the heartbeat mechanism to improve the reliability of master-slave replication

In the Redis master-slave replication mode, if there is data synchronization between the master and slave servers, the slave server will send a REPLCONF ACK command to the master server once a second by default to ensure that the connection between the two is smooth. This mechanism of periodic interactive commands to ensure connection is called the "heartbeat" mechanism. In the command line of the redis-master master server opened above, if you run the info replication command, you can see the "heartbeat" status of its slave servers.

 127.0.0.1:6379> info replication
 2 # Replication
 3 role:master
 4 connected_slaves:2
 5 slave0:ip=172.17.0.3,port=6380,state=online,offset=16185,lag=1
 6 slave1:ip=172.17.0.4,port=6381,state=online,offset=16185,lag=1

In lines 5 and 6, the time it takes for the slave server to send the REPLCONF ACK command can be represented by lag, which is 1 second, indicating that the connection between the two slave servers and the master server is smooth.

Here you can imagine that if the slave server goes down, then master-slave replication will be meaningless. To this end, the following steps can be used to associate the heartbeat mechanism with the active replication action.

The first step is to create a new redisMaster.conf file in the C:\work\redis\redisConf directory and write the following code in it.

min-slaves-to-write 2

min-slaves-max-lag 15

The parameters in the first line indicate that the minimum number of slave servers required to implement master-slave replication is 2. The parameters in the second line indicate that if the heartbeat delay (i.e., lag value) of the number of slave servers specified by the parameters in the first line (here 2) is greater than 15 seconds, master-slave replication will not be performed.

These two conditions are in an "or" relationship, that is, as long as the number of slave servers is less than 2, or the heartbeat delay between the two slave servers is greater than 15 seconds, the master server stops the master-slave replication operation.

In the second step, start the redis-master container with the following command. Since the above configuration has been loaded when the Redis server is started at this time, the Redis master server will detect the conditions set in the first step when performing master-slave replication. This can improve the reliability of master-slave replication.

docker run -itd --name redis-master -v C:\work\redis\redisConf:/redisConfig:rw -p 6379:6379 redis:latest redis-server /redisConfig/redisMaster.conf

6 Use offset to check data consistency

In the command line of the redis-master master server opened above, if you run the info replication command, you can also see the master_repl_offset data representing the offset of the replicated data, as shown in line 6 below. The data here is 276, which indicates the number of bytes of data sent by the master server to the slave server.

127.0.0.1:6379> info replication
 # Replication
 role:master
 connected_slaves:1
 …
 master_repl_offset:276

Similarly, if you go to the command line of the redis-slave1 slave server, you can also view the offset through info replication, as shown in line 7 below.

127.0.0.1:6380> info replication
 # Replication
 role:slave
 master_host:172.17.0.2
 master_port:6379
 …
 slave_repl_offset:276

In the slave server, this data indicates the number of data bytes received from the master server. If the data in the master and slave servers are consistent, it means that the data between the master and slave servers are synchronized.

After running the set nextVal 1 command in the master server redis-master, and then using info replication to view the master_repl_offset value, you will find that there is a change. At this time, run the info replication command on the redis-slave1 slave server, and you will find that the master_repl_offset value of the slave server is still consistent with that of the master server. This shows that the data added in the master server using the set nextVal 1 command has been successfully synchronized to the slave server. In other words, if a Redis problem occurs, you can use the master_repl_offset value to check whether the synchronized data is correct, and then further troubleshoot the problem.

Summarize

This is the end of this article about using Docker to build a Redis master-slave replication cluster. For more information about using Docker to build a Redis master-slave replication cluster, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!