MySQL partition table is classified by month

MySQL partition table is classified by month

It is recommended that the data volume of a single MySQL table should not exceed 20 million rows, otherwise it will have a significant impact on performance. I recently took over a project with a single table containing over 70 million rows of data. A simple query took more than 50 minutes to produce any results, which was really frustrating. In the end, we decided to use a partitioned table.

Create a table

After a general table (innodb) is created, there is only one idb file:

create table normal_table(id int primary key, no int)

View the database file:

normal_table.ibd  

Create a partition table partitioned by month, attention! In addition to the regular primary key, the month field (the field used for partitioning) must also be the primary key:

create table partition_table(id int AUTO_INCREMENT, create_date date, name varchar(10), 
primary key(id, create_date)) ENGINE=INNODB DEFAULT CHARSET=utf8 
partition by range(month(create_date))(
partition quarter1 values ​​less than(4),
partition quarter2 values ​​less than(7),
partition quarter3 values ​​less than(10),
partition quarter4 values ​​less than(13)
);

View the database file:

partition_table#p#quarter1.ibd  
partition_table#p#quarter2.ibd  
partition_table#p#quarter3.ibd  
partition_table#p#quarter4.ibd

insert

insert into partition_table(create_date, name) values("2021-01-25", "tom1");
insert into partition_table(create_date, name) values("2021-02-25", "tom2");
insert into partition_table(create_date, name) values("2021-03-25", "tom3");
insert into partition_table(create_date, name) values("2021-04-25", "tom4");
insert into partition_table(create_date, name) values("2021-05-25", "tom5");
insert into partition_table(create_date, name) values("2021-06-25", "tom6");
insert into partition_table(create_date, name) values("2021-07-25", "tom7");
insert into partition_table(create_date, name) values("2021-08-25", "tom8");
insert into partition_table(create_date, name) values("2021-09-25", "tom9");
insert into partition_table(create_date, name) values("2021-10-25", "tom10");
insert into partition_table(create_date, name) values("2021-11-25", "tom11");
insert into partition_table(create_date, name) values("2021-12-25", "tom12");

Query

select count(*) from partition_table;
> 12

 
Query the data of the second partition (second quarter):
select * from partition_table PARTITION(quarter2);

4 2021-04-25 tom4
5 2021-05-25 tom5
6 2021-06-25 tom6

delete

When a table is deleted, all partition files of the table will be deleted.

Supplement: Mysql automatically partitions by month table

The core two stored procedures:

  • auto_create_partition is used to create table partitions. After being called, table partitions are created for the table until the end of the next month.
  • auto_del_partition is used to delete table partitions to facilitate the recovery of historical data space.
DELIMITER $$
DROP PROCEDURE IF EXISTS auto_create_partition$$
CREATE PROCEDURE `auto_create_partition`(IN `table_name` varchar(64))
BEGIN
   SET @next_month:=CONCAT(date_format(date_add(now(),interval 2 month),'%Y%m'),'01');
   SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '`',
     ' ADD PARTITION (PARTITION p', @next_month, " VALUES LESS THAN (TO_DAYS(",
       @next_month ,")) );" );
   PREPARE STMT FROM @SQL;
   EXECUTE STMT;
   DEALLOCATE PREPARE STMT;
END$$

DROP PROCEDURE IF EXISTS auto_del_partition$$
CREATE PROCEDURE `auto_del_partition`(IN `table_name` varchar(64), IN `reserved_month` int)
BEGIN
 DECLARE v_finished INTEGER DEFAULT 0;
 DECLARE v_part_name varchar(100) DEFAULT "";
 DECLARE part_cursor CURSOR FOR 
  select partition_name from information_schema.partitions where table_schema = schema()
   and table_name=@table_name and partition_description < TO_DAYS(CONCAT(date_format(date_sub(now(),interval reserved_month month),'%Y%m'),'01'));
 DECLARE continue handler FOR 
  NOT FOUND SET v_finished = TRUE;
 OPEN part_cursor;
read_loop: LOOP
 FETCH part_cursor INTO v_part_name;
 if v_finished = 1 then
  leave read_loop;
 end if;
 SET @SQL = CONCAT( 'ALTER TABLE `', table_name, '` DROP PARTITION ', v_part_name, ";" );
 PREPARE STMT FROM @SQL;
 EXECUTE STMT;
 DEALLOCATE PREPARE STMT;
 END LOOP;
 CLOSE part_cursor;
END$$

DELIMITER ;

Here is an example

-- Assume there is a table called records, set the partition condition to partition by month by end_time DROP TABLE IF EXISTS `records`;
CREATE TABLE `records` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `start_time` datetime NOT NULL,
  `end_time` datetime NOT NULL,
  `memo` varchar(128) CHARACTER SET utf8mb4 NOT NULL,
  PRIMARY KEY (`id`,`end_time`)
) 
PARTITION BY RANGE (TO_DAYS(end_time))(
 PARTITION p20200801 VALUES LESS THAN ( TO_DAYS('20200801'))
);

DROP EVENT IF EXISTS `records_auto_partition`;

-- Create an Event that executes once a month and saves up to 6 months of data DELIMITER $$
CREATE EVENT `records_auto_partition`
ON SCHEDULE EVERY 1 MONTH ON COMPLETION PRESERVE
ENABLE
DO
BEGIN
call auto_create_partition('records');
call auto_del_partition('records',6);
END$$
DELIMITER ;

A few notes:

  • For MySQL 5.1 and above, the index field of the table partition must be the primary key.
  • In a stored procedure, DECLARE must be followed by BEGIN, otherwise an incomprehensible error will be reported.
  • The cursor DECLARE must be after the definition statement, otherwise an error will be reported
  • If you installed MySQL yourself, the Event function may not be enabled, and an error message will be displayed when creating an Event. Modify my.cnf, add event_scheduler=1 under [mysqld], and restart.

This is the end of this article about how to implement monthly classification in MySQL partitioned tables. For more information about MySQL partitioned tables by month, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Why must the partition key in a MySQL partition table be part of the primary key?
  • A performance bug about MySQL partition tables
  • Detailed explanation of MySQL partition table
  • Management and maintenance of Mysql partition tables
  • MySQL optimization partition table
  • Summary of MySQL partition table management commands

<<:  SystemC environment configuration method under Linux system

>>:  HTML form tag usage learning tutorial

Recommend

How to recover files accidentally deleted by rm in Linux environment

Table of contents Preface Is there any hope after...

Example of Html shielding right-click menu and left-click typing function

Disable right-click menu <body oncontextmenu=s...

Beginners learn some HTML tags (3)

Related articles: Beginners learn some HTML tags ...

Summary and practice of javascript prototype chain diagram

Table of contents Prototype chain We can implemen...

How to publish static resources in nginx

step Place the prepared static resource files in ...

Example code for implementing complex table headers in html table

Use HTML to create complex tables. Complex tables...

Example code for implementing anti-shake in Vue

Anti-shake: Prevent repeated clicks from triggeri...

...

MySQL parameter related concepts and query change methods

Preface: In some previous articles, we often see ...

How to query and update the same table in MySQL database at the same time

In ordinary projects, I often encounter this prob...

Application of Beautiful Style Sheets in XHTML+CSS Web Page Creation

This is an article written a long time ago. Now it...

JavaScript Dom implements the principle and example of carousel

If we want to make a carousel, we must first unde...

Hexadecimal color codes (full)

Red and pink, and their hexadecimal codes. #99003...

Solve the MySQL 5.7.9 version sql_mode=only_full_group_by problem

MySQL 5.7.9 version sql_mode=only_full_group_by i...