A brief discussion on three methods of asynchronous replication in MySQL 8.0

A brief discussion on three methods of asynchronous replication in MySQL 8.0

In this experiment, we configure MySQL standard asynchronous replication with one master and two slaves for three modes: empty database, offline, and online. Only the entire server level replication is performed, and individual database tables or filtered replication are not considered.

Experimental environment

[root@slave2 ~]# cat /etc/hosts
192.168.2.138 master
192.168.2.192 slave1
192.168.2.130 slave2
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.16 |
+-----------+
1 row in set (0.00 sec)

1. Empty warehouse

1. View the main library binary information

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000004 | 155 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

2. Create a replication user on the master database

mysql> create user 'repl'@'%' identified with mysql_native_password by 'wwwwww';
Query OK, 0 rows affected (0.03 sec)
mysql> grant replication client,replication slave on *.* to 'repl'@'%';
Query OK, 0 rows affected (0.04 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

3. Create master database information from the slave database

mysql> stop slave;
mysql> change master to master_host='192.168.2.138', master_port=3306, master_user='repl', master_password='wwwwww', master_log_file='mysql-bin.000004', master_log_pos=155;
Query OK, 0 rows affected, 2 warnings (0.00 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G

We did not create a repl user on the slave database just now, but because the create user statement is executed after the starting point, it can be copied to the slave database normally. You can confirm this by querying the mysql.user table.

sql> select * from mysql.user where user='repl'\G

2. Offline

If the database already has application data, but allows an acceptable offline time window for replication, a common approach in this scenario is to directly copy the entire data directory of the master database to the slave database, and then start replication. The specific steps are as follows.

1. Create a test library and test table on the master node

CREATE DATABASE test;
Query OK, 1 row affected (0.04 sec)
mysql> USE test;
Database changed
mysql> CREATE TABLE t(id int(10));
Query OK, 0 rows affected (0.09 sec)
mysql> INSERT INTO t VALUES (111);
Query OK, 1 row affected (0.05 sec)
mysql> INSERT INTO t VALUES (222);
Query OK, 1 row affected (0.00 sec)
mysql> INSERT INTO t VALUES (333);
Query OK, 1 row affected (0.00 sec)

2. Create a replication user in the master database

mysql> create user 'repl'@'%' identified with mysql_native_password by 'wwwwww';
Query OK, 0 rows affected (0.03 sec)
mysql> grant replication client,replication slave on *.* to 'repl'@'%';
Query OK, 0 rows affected (0.04 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

3. Stop all replicated instances and execute on master, slave1, and slave2 respectively

[root@master ~]# ln -s /usr/local/mysql/bin/mysqladmin /usr/bin/mysqladmin
[root@master ~]# mysqladmin -hlocalhost -uroot -pwwwwww shutdown

4. Copy data to slave1 and slave2

[root@master data]# cd /data
[root@master data]# scp -r mysql/ slave1:/data/
[root@master data]# scp -r mysql/ slave2:/data/

5. Execute commands from slave1 and slave2 to delete the auto.cnf file

[root@slave1 mysql]# cd /data/mysql
[root@slave1 mysql]# rm -rf auto.cnf
[root@slave2 mysql]# cd /data/mysql
[root@slave2 mysql]# rm -rf auto.cnf
 

6. Restart the instance. This needs to be done on all three nodes.

[root@master data]# service mysqld start
Starting MySQL.. SUCCESS!

7. View the binary log in the main library

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000005 | 155 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

8. Execute commands from slave1 and slave2

mysql> stop slave;
Query OK, 0 rows affected (0.01 sec)
mysql> change master to master_host='192.168.2.138', master_port=3306, master_user='repl', master_password='wwwwww', master_log_file='mysql-bin.000005', master_log_pos=155;
Query OK, 0 rows affected, 2 warnings (0.02 sec)
mysql> start slave;
Query OK, 0 rows affected (0.03 sec)
mysql> show slave status\G

9. Execute commands from slave1 and slave2 to check whether the database and table are synchronized.

mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| t |
+----------------+
1 row in set (0.00 sec)
mysql> select * from t;
+------+
|id|
+------+
| 111 |
| 222 |
| 333 |
+------+
3 rows in set (0.00 sec)

3. mysqldump online

The requirement to establish replication offline is too idealistic. In most cases, replication is required to be created online without affecting online business, and the impact on the online database is required to be as small as possible. For example, locking the master database during replication affects access to the master database and is therefore usually not allowed. There are two alternative replication solutions for this scenario: using the mysqldump program or using a third-party tool such as XtraBackup. These two solutions have their own applicable occasions. The process of establishing replication online using mysqldump is as follows.

1. Create a test database and table in the main library

mysql> CREATE DATABASE test;
Query OK, 1 row affected (0.04 sec)
mysql> use test;
Database changed
mysql> CREATE TABLE t(id int(10));
Query OK, 0 rows affected (0.10 sec)
mysql> INSERT INTO t VALUES(111);
Query OK, 1 row affected (0.09 sec)
mysql> INSERT INTO t VALUES(222);
Query OK, 1 row affected (0.00 sec)
mysql> INSERT INTO t VALUES(333);
Query OK, 1 row affected (0.05 sec)
mysql> INSERT INTO t VALUES(444);
Query OK, 1 row affected (0.00 sec)

2. Create a replication user in the master database

mysql> create user 'repl'@'%' identified with mysql_native_password by 'wwwwww';
Query OK, 0 rows affected (0.01 sec)
mysql> grant replication client,replication slave on *.* to 'repl'@'%';
Query OK, 0 rows affected (0.01 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
 

3. Create master database information in slave1 and slave2

mysql> change master to master_host='192.168.2.138', master_port=3306, master_user='repl', master_password='wwwwww';
Query OK, 0 rows affected, 2 warnings (0.04 sec)

4. Use the mysqldump command to copy data from the slave1 and slave2 libraries

[root@slave2 ~]# mysqldump --single-transaction --all-databases --master-data=1 --host=192.168.2.138 --user=root --password=wwwwww --apply-slave-statements | mysql -uroot -pwwwwww -hlocalhost
mysql: [Warning] Using a password on the command line interface can be insecure.
mysqldump: [Warning] Using a password on the command line interface can be insecure.

Parameter Description

The –single-transaction parameter enables non-locking exports of Innodb tables. This option sets the transaction isolation mode to REPEATABLE READ and sends a START TRANSACTION SQL statement to the server before dumping data. It only works with transactional tables like Innodb, as it dumps a consistent state of the database when START TRANSACTION is issued, without blocking any applications. Therefore, it is assumed here that: 1. All application data tables use the Innodb engine. 2. All system table data will not change during the backup process.

The --master-data parameter causes the dump output to include a SQL statement similar to CHANGE MASTER TO MASTER_LOG_FILE='binlog.000004', MASTER_LOG_POS=1480; , which indicates the binary log coordinates (file name and position) of the master. If the option value is 2, the CHANGE MASTER TO statement is written as an SQL comment and is therefore only informative and not executed. If the parameter value is 1, the statement is not written as a comment and is executed when the dump file is reloaded. If no option value is specified, the default value is 1.

The –apply-slave-statements parameter adds a STOP SLAVE statement before the CHANGE MASTER TO statement and a START SLAVE statement at the end of the output to automatically start replication.

Through the pipeline operator, export and import are performed in one step without the need to generate files in the middle.

5. Confirm the replication status from the database

mysql> show slave status\G

6. Check whether the database and table are copied successfully from the database

use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from t;
+------+
|id|
+------+
| 111 |
| 222 |
| 333 |
| 444 |
| 555 |
+------+
5 rows in set (0.00 sec)

The advantage of the mysqldump method is that it can perform partial replication. For example, if you define replicate-do-table=db1.* in the configuration file, you can use this method to replicate only the db1 library and ignore other replication events. The disadvantage is that mysqldump will generate SQL statements for dumping data from the main database. It is actually a logical backup method, so it is slow and not suitable for large databases.

4. XtraBackup online replication

Another option for establishing replication online is to use XtraBackup. XtraBackup is an open source project of Percona, which is used to implement functions similar to InnoDB Hot Backup, the official hot backup tool of InnoDB. It supports online hot backup and does not affect data reading and writing during backup. So far, the latest version is Percona XtraBackup 8.0.6, which can be downloaded from https://www.percona.com/downloads/. XtraBackup has many features and advantages, such as support for full backup, incremental backup, partial backup; support for compressed backup; backup does not affect data reading and writing, transactions, etc., but it also has defects and shortcomings: for example, it does not support offline backup, does not support direct backup to tape devices, does not support Cloud Back, and MyISAM backup will also be blocked. However, these minor flaws do not affect XtraBackup from becoming a popular MySQL backup tool. In addition, please note that XtraBackup only supports Linux platforms, not Windows platforms. The following demonstrates the process of setting up master-slave replication online with XtraBackup. The master database has established the user repl for performing replication.

Create a replication user in the master database

mysql> create user 'repl'@'%' identified with mysql_native_password by 'wwwwww';
Query OK, 0 rows affected (0.01 sec)
mysql> grant replication client,replication slave on *.* to 'repl'@'%';
Query OK, 0 rows affected (0.01 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)

1. Install XtraBackupv on both the master and slave libraries

[root@master ~]# yum -y install libev
[root@master home]# yum localinstall percona-xtrabackup-80-8.0.6-1.el7.x86_64.rpm -y

2. Configure SSH password-free connection from the master library to the slave library

[root@master home]# ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:GBLbrw17UVck8RyCa/fbYyLkSNZIRc5p+jPQmpkD+bI root@master
The key's randomart image is:
+---[RSA 2048]----+
| . .o+o+ |
| + +..* . |
| ooo*.o |
| . +.o*.. |
|ooS+oo . |
| =o=Bo . |
| o.=B++ o |
| .o..oo..o.|
| E . o .|
+----[SHA256]-----+
[root@master home]# ssh-copy-id 192.168.2.138
[root@master home]# ssh-copy-id 192.168.2.192
[root@master home]# ssh-copy-id 192.168.2.130

3. Stop the slave database and delete the data in the slave database

[root@slave1 home]# service mysql stop
[root@slave2 home]# service mysql stop
[root@slave1 home]# rm -rf /data/mysql/*
[root@slave2 home]# rm -rf /data/mysql/*

4.Back up data and transfer

[root@master tmp]# xtrabackup -uroot -pwwwwww --socket=/data/mysql/mysql.sock --no-lock --backup --compress --stream=xbstream --parallel=4 --target-dir=./ | ssh root@192.168.2.192 "xbstream -x -C /data/mysql/ --decompress"

An error occurs during execution.

190606 01:21:47 >> log scanned up to (19597291)

190606 01:21:47 Selecting LSN and binary log position from p_s.log_status

Error: failed to fetch query result SELECT server_uuid, local, replication, storage_engines FROM performance_schema.log_status: Access denied; you need (at least one of) the BACKUP_ADMIN privilege(s) for this operation

mysql> grant BACKUP_ADMIN on *.* to 'root'@'%';
Query OK, 0 rows affected (0.01 sec)

Run the following command to delete the content of 192.168.2.192:/data/mysql/* and execute the command again to find that it is correct. A successful execution looks like this:


This command connects to the master, performs a parallel compressed streaming backup, transfers the backup to the slave through the pipe operator, and decompresses it directly into the slave's data directory. All operations are completed with one command, without the need to write files to disk in the middle.

5. Restore the backup from the library

[root@slave1 /]# xtrabackup --prepare --target-dir=/data/mysql
[root@slave2 /]# xtrabackup --prepare --target-dir=/data/mysql

6. View the binary bin-log log from the library

[root@slave1 mysql]# cat xtrabackup_binlog_info 
mysql-bin.000008 155
[root@slave2 mysql]# cat xtrabackup_binlog_info 
mysql-bin.000009 155

7. Start the slave library

[root@slave1 data]# service mysqld start
Starting MySQL... SUCCESS! 
[root@slave2 data]# service mysqld start
Starting MySQL... SUCCESS!

8. Create the master database information, where the master_log_file and master_log_pos values ​​come from step 6

mysql> change master to master_host='192.168.2.138', master_port=3306, master_user='repl', master_password='wwwwww', master_log_file='mysql-bin.000008', master_log_pos=155;
Query OK, 0 rows affected, 2 warnings (0.04 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G

9. Test data from the library

mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from t;
+------+
|id|
+------+
| 111 |
| 222 |
| 333 |
| 444 |
| 555 |
+------+
5 rows in set (0.00 sec)

XtraBackup is a physical copy with much higher performance than mysqldump and minimal impact on the master database. It is very suitable for creating a high-load, large-data-volume, full-instance slave database online from scratch.

This concludes this article on the three methods of MySQL 8.0 asynchronous replication. For more information about MySQL 8.0 asynchronous replication, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Detailed explanation of MySQL master-slave replication and read-write separation
  • How to copy MySQL table
  • Automatic failover of slave nodes in replication architecture in MySQL 8.0.23
  • MYSQL database GTID realizes master-slave replication (super convenient)
  • Implementation principle and configuration of MySql master-slave replication
  • A brief analysis of MySQL's WriteSet parallel replication
  • MySQL master-slave replication principle and points to note
  • How to dynamically modify the replication filter in mysql
  • A brief analysis of MySQL parallel replication
  • How to skip errors in mysql master-slave replication
  • MySQL 4 common master-slave replication architectures
  • Solution to MySQL replication failure caused by disk fullness

<<:  Analysis of the Linux input subsystem framework principle

>>:  JavaScript implements simple calculator function

Recommend

Vue two same-level components to achieve value transfer

Vue components are connected, so it is inevitable...

Example of using CSS to achieve semi-transparent background and opaque text

This article introduces an example of how to use ...

Advantages of MySQL covering indexes

A common suggestion is to create indexes for WHER...

Native js implementation of slider interval component

This article example shares the specific code of ...

Common CSS Errors and Solutions

Copy code The code is as follows: Difference betw...

Detailed explanation of using Vue custom tree control

This article shares with you how to use the Vue c...

vue+springboot realizes login function

This article example shares the specific code of ...

Docker builds python Flask+ nginx+uwsgi container

Install Nginx First pull the centos image docker ...

A summary of detailed insights on how to import CSS

The development history of CSS will not be introd...

Basic usage of @Font-face and how to make it compatible with all browsers

@Font-face basic introduction: @font-face is a CSS...

What is ZFS? Reasons to use ZFS and its features

History of ZFS The Z File System (ZFS) was develo...

Understanding and example code of Vue default slot

Table of contents What is a slot Understanding of...