Detailed explanation of mysql replication tool based on python

1. Introduction
Second practice

2.1 Installation and Configuration
2.2 Core Class Introduction
2.3 How to use it?
2.3 Extension

1. Introduction

python-mysql-replication is a MySQL replication protocol tool implemented in Python. We can use it to parse binlog to obtain insert, update, delete and other events in the log, and meet other business needs based on this. For example, when data changes, the cache is invalidated, and DML events are listened to to notify downstream business parties to take corresponding actions.

Project information

Second practice

2.1 Installation and Configuration

Get the source code

git clone http://www.github.com/noplay/python-mysql-replication

Install using pip

pip install mysql-replication

Permissions: You can use the replication account directly or use other accounts, but the account must have SELECT, REPLICATION SLAVE, REPLICATION CLIENT permissions

GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON . TO 'replicator'@'%' IDENTIFIED BY 'xxxxx';

The parameter settings related to database log are as follows:

log_bin=on,binlog_format=row,binlog_row_image=FULL

2.2 Core Class Introduction

The entry point of python-mysql-replication is the class BinLogStreamReader(). When using this tool, we need to instantiate a BinLogStreamReader() object stream. BinLogStreamReader registers as a slave role to the master database through ReportSlave to receive MySQL binlog broadcasts. Those who are interested can study the specific implementation of its code.

This instance provides a collection of various events for parsing binlog, and each event is also an object.

The parameters required to initialize the BinLogStreamReader() instance are as follows:

2.3 How to use it?

The simplest usage script name is pyreplica.py

Open two windows, one for execution and the other for mysql to write or modify data

python pyreplica.py

The output is as follows:

2.3 Extension

Based on the log event analysis provided by this tool, we can do a lot of things. The more famous tool binlog2sql uses this tool to parse binlog for data rollback.

mysql-replication.py

The result of executing the script is as follows

In addition to parsing binlog, we can also use thon-mysql-replication to perform full and incremental data migration. For example, it can be used when you only want to migrate some large tables instead of the entire database. Friends who are interested can think about the approximate algorithm.

Summarize

The above is the Python-based MySQL replication tool introduced by the editor. I hope it will be helpful to everyone. If you have any questions, please leave me a message and the editor will reply to you in time. I would also like to thank everyone for their support of the 123WORDPRESS.COM website!
If you find this article helpful, please feel free to reprint it and please indicate the source. Thank you!

You may also be interested in:

MySQL management tool code examples written in Python
Python implements a small tool to read Excel and write to MySQL
Sharing simple steps of Python operating MySQL database
Steps for Python crawler to crawl global epidemic data and store it in MySQL database
Python crawls Tencent's real-time epidemic data and stores it in MySQL database sample code
Solve the problem of python mysql insert statement
Python inserts null values into mysql
Python implements a script to synchronize MySQL specified table incremental data to clickhouse
Develop a small tool to operate MySQL using Python

<<: Two implementation solutions for vuex data persistence

>>: How to install theano and keras on ubuntu system