MySQL partition table is classified by month

MySQL partition table is classified by month

It is recommended that the data volume of a single MySQL table should not exceed 20 million rows, otherwise it will have a significant impact on performance. I recently took over a project with a single table containing over 70 million rows of data. A simple query took more than 50 minutes to produce any results, which was really frustrating. In the end, we decided to use a partitioned table.

Create a table

After a general table (innodb) is created, there is only one idb file:

create table normal_table(id int primary key, no int)

View the database file:

normal_table.ibd  

Create a partition table partitioned by month, attention! In addition to the regular primary key, the month field (the field used for partitioning) must also be the primary key:

create table partition_table(id int AUTO_INCREMENT, create_date date, name varchar(10), 
primary key(id, create_date)) ENGINE=INNODB DEFAULT CHARSET=utf8 
partition by range(month(create_date))(
partition quarter1 values ​​less than(4),
partition quarter2 values ​​less than(7),
partition quarter3 values ​​less than(10),
partition quarter4 values ​​less than(13)
);

View the database file:

partition_table#p#quarter1.ibd  
partition_table#p#quarter2.ibd  
partition_table#p#quarter3.ibd  
partition_table#p#quarter4.ibd

insert

insert into partition_table(create_date, name) values("2021-01-25", "tom1");
insert into partition_table(create_date, name) values("2021-02-25", "tom2");
insert into partition_table(create_date, name) values("2021-03-25", "tom3");
insert into partition_table(create_date, name) values("2021-04-25", "tom4");
insert into partition_table(create_date, name) values("2021-05-25", "tom5");
insert into partition_table(create_date, name) values("2021-06-25", "tom6");
insert into partition_table(create_date, name) values("2021-07-25", "tom7");
insert into partition_table(create_date, name) values("2021-08-25", "tom8");
insert into partition_table(create_date, name) values("2021-09-25", "tom9");
insert into partition_table(create_date, name) values("2021-10-25", "tom10");
insert into partition_table(create_date, name) values("2021-11-25", "tom11");
insert into partition_table(create_date, name) values("2021-12-25", "tom12");

Query

select count(*) from partition_table;
> 12

 
Query the data of the second partition (second quarter):
select * from partition_table PARTITION(quarter2);

4 2021-04-25 tom4
5 2021-05-25 tom5
6 2021-06-25 tom6

delete

When a table is deleted, all partition files of the table will be deleted.

Supplement: Mysql automatically partitions by month table

The core two stored procedures:

  • auto_create_partition is used to create table partitions. After being called, table partitions are created for the table until the end of the next month.
  • auto_del_partition is used to delete table partitions to facilitate the recovery of historical data space.
DELIMITER $$
DROP PROCEDURE IF EXISTS auto_create_partition$$
CREATE PROCEDURE `auto_create_partition`(IN `table_name` varchar(64))
BEGIN
   SET @next_month:=CONCAT(date_format(date_add(now(),interval 2 month),'%Y%m'),'01');
   SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '`',
     ' ADD PARTITION (PARTITION p', @next_month, " VALUES LESS THAN (TO_DAYS(",
       @next_month ,")) );" );
   PREPARE STMT FROM @SQL;
   EXECUTE STMT;
   DEALLOCATE PREPARE STMT;
END$$

DROP PROCEDURE IF EXISTS auto_del_partition$$
CREATE PROCEDURE `auto_del_partition`(IN `table_name` varchar(64), IN `reserved_month` int)
BEGIN
 DECLARE v_finished INTEGER DEFAULT 0;
 DECLARE v_part_name varchar(100) DEFAULT "";
 DECLARE part_cursor CURSOR FOR 
  select partition_name from information_schema.partitions where table_schema = schema()
   and table_name=@table_name and partition_description < TO_DAYS(CONCAT(date_format(date_sub(now(),interval reserved_month month),'%Y%m'),'01'));
 DECLARE continue handler FOR 
  NOT FOUND SET v_finished = TRUE;
 OPEN part_cursor;
read_loop: LOOP
 FETCH part_cursor INTO v_part_name;
 if v_finished = 1 then
  leave read_loop;
 end if;
 SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '` DROP PARTITION ', v_part_name, ";" );
 PREPARE STMT FROM @SQL;
 EXECUTE STMT;
 DEALLOCATE PREPARE STMT;
 END LOOP;
 CLOSE part_cursor;
END$$

DELIMITER ;

Here is an example

-- Assume there is a table called records, set the partition condition to partition by month by end_time DROP TABLE IF EXISTS `records`;
CREATE TABLE `records` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `start_time` datetime NOT NULL,
  `end_time` datetime NOT NULL,
  `memo` varchar(128) CHARACTER SET utf8mb4 NOT NULL,
  PRIMARY KEY (`id`,`end_time`)
) 
PARTITION BY RANGE (TO_DAYS(end_time))(
 PARTITION p20200801 VALUES LESS THAN ( TO_DAYS('20200801'))
);

DROP EVENT IF EXISTS `records_auto_partition`;

-- Create an Event that executes once a month and saves up to 6 months of data DELIMITER $$
CREATE EVENT `records_auto_partition`
ON SCHEDULE EVERY 1 MONTH ON COMPLETION PRESERVE
ENABLE
DO
BEGIN
call auto_create_partition('records');
call auto_del_partition('records',6);
END$$
DELIMITER ;

A few notes:

  • For MySQL 5.1 and above, the index field of the table partition must be the primary key.
  • In a stored procedure, DECLARE must be followed by BEGIN, otherwise an incomprehensible error will be reported.
  • The cursor DECLARE must be after the definition statement, otherwise an error will be reported
  • If you installed MySQL yourself, the Event function may not be enabled, and an error message will be displayed when creating an Event. Modify my.cnf, add event_scheduler=1 under [mysqld], and restart.

This is the end of this article about how to implement monthly classification in MySQL partitioned tables. For more information about MySQL partitioned tables by month, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Why must the partition key in a MySQL partition table be part of the primary key?
  • A performance bug about MySQL partition tables
  • Detailed explanation of MySQL partition table
  • Management and maintenance of Mysql partition tables
  • MySQL optimization partition table
  • Summary of MySQL partition table management commands

<<:  SystemC environment configuration method under Linux system

>>:  HTML form tag usage learning tutorial

Recommend

How to install and deploy zabbix 5.0 for nginx

Table of contents Experimental environment Instal...

HTML page jump and parameter transfer issues

HTML page jump: window.open(url, "", &q...

Analyzing Linux high-performance network IO and Reactor model

Table of contents 1. Introduction to basic concep...

MySQL 8.0.12 installation configuration method and password change

This article records the installation and configu...

How to view the status of remote server files in Linux

As shown below: The test command determines wheth...

Native js to implement form validation function

Table of contents When developing, analyzing the ...

Detailed explanation of samba + OPENldap to build a file sharing server

Here I use samba (file sharing service) v4.9.1 + ...

Vue implements user login and token verification

In the case of complete separation of the front-e...

XHTML: Frame structure tag

Frame structure tag <frameset></frameset...

How to implement email alert in zabbix

Implemented according to the online tutorial. zab...

MySQL log system detailed information sharing

Anyone who has worked on a large system knows tha...

Detailed explanation of JS browser event model

Table of contents What is an event A Simple Examp...

Vue login function implementation

Table of contents Written in front Login Overview...