Detailed explanation of non-primary key column overflow monitoring in MySQL tables

Detailed explanation of non-primary key column overflow monitoring in MySQL tables

Today, I fell into the trap again. I have encountered MySQL primary key overflow before, and I monitored it through prometheus. For details, see this MySQL primary key overflow review

The pitfalls encountered this time were more hidden. A column of type int signed in a log table is full. The quick solution is of course to switch to a new table as an emergency measure, and then move some historical data from the old table to the hot table.

It’s time to mend the fold after the failure. After resolving the issue, quickly write a script to review all other production tables.

The following is a detection script I am using temporarily. It is not perfect yet, so I will use it as it is.

Divide into 2 files (1 sql file, 1 shell script)

The content of check.sql is as follows:

SELECT 
cast( pow(2, case data_type
  when 'tinyint' then 7
  when 'smallint' then 15
  when 'mediumint' then 23
  when 'int' then 31
  when 'bigint' then 63
  end+(column_type like '% unsigned'))-1 as decimal(30,0)) as max_int,
' - ',
concat ('(', concat('select ','max(',COLUMN_NAME,')',' from ',TABLE_SCHEMA,'.',TABLE_NAME),')') 
from 
information_schema.COLUMNS 
where 
TABLE_SCHEMA NOT IN ('information_schema','sys','test','mysql','performance_schema') 
AND 
 DATA_TYPE IN ('int' ) ;

Execute directly in the database, the effect is similar to this:

The content of check.sh is as follows:

#!/bin/bash
# Monitor int type. When the available space is less than 5 million, remind to do DDL operation. # Set max_execution_time of session level to 2 seconds to prevent large columns without index from slowing down the database. However, this may miss some columns, so please pay attention. # Note: I have not checked bigint type here. If necessary, please modify DATA_TYPE in check.sql where condition to add bigint check source /etc/profile
set -u

mkdir $(date +%F) -pv

# step1 detection for host in {'192.168.1.100','192.168.1.110','192.168.1.120','192.168.1.130'}; do

mysql -udts -pdts -h${host} -BN < check.sql 2>/dev/null > sql.log
wait

echo "Description: | Maximum value allowed for the current column | SQL for inspection " >> $(date +%F)/$host.log

while read line; do
  ret=$(mysql -udts -pdts -h${host} -BNe "set session max_execution_time=2000;select $line" 2>/dev/null)
  echo ${ret}
  if [[ "${ret}" == "NULL" ]]; then
  continue
  fi
  if [ ${ret} -lt 5000000 ] ; then 
   echo "$line remaining space ${ret}, the table available water level is less than 5 million, it is recommended to modify the DDL to bigint type" >> $(date +%F)/$host.log
  
  fi
done < ./sql.log

done

# step2 Package the inspection contents and send them by email (you may need to modify this according to your production situation)
tar czf $(date +%F).tar.gz $(date +%F)
sendemail -s 192.168.1.200 -f [email protected] -t [email protected] -a $(date +%F).tar.gz -u "$(date +%F) int water level inspection log" -o message-content-type=html -o message-charset=utf8 -m "For details, please see the attachment"

# step3 clean up the date-named directories and tar.gz files generated daily. I won’t post the commands here.

Then add a cronjob at 10 am every day.

Finally, the content of the emails received every day is roughly similar to the following:

This is the end of this article on how to monitor the overflow of non-primary key columns in MySQL tables. For more information about MySQL non-primary key column overflows, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • In-depth discussion on auto-increment primary keys in MySQL
  • The difference and advantages and disadvantages of Mysql primary key UUID and auto-increment primary key
  • MySQL primary key naming strategy related
  • What to do if the auto-increment primary key in MySQL is used up
  • Solution to running out of MySQL's auto-increment ID (primary key)
  • In-depth analysis of why MySQL does not recommend using uuid or snowflake id as primary key
  • Use prometheus to count the remaining available percentage of MySQL auto-increment primary keys
  • Mysql SQL statement operation to add or modify primary key

<<:  Implementation of k8s deployment of docker container

>>:  Method of dynamically loading geojson based on Vue+Openlayer

Recommend

W3C Tutorial (16): Other W3C Activities

This section provides an overview of some other i...

How to periodically clean up images that are None through Jenkins

Preface In the process of continuous code deliver...

Implementation of mysql decimal data type conversion

Recently, I encountered a database with the follo...

MySQL stored procedure method example of returning multiple values

This article uses an example to describe how to r...

uniapp realizes the recording upload function

Table of contents uni-app Introduction HTML part ...

Summary of the use of Datetime and Timestamp in MySQL

Table of contents 1. How to represent the current...

How to implement rounded corners with CSS3 using JS

I found an example when I was looking for a way t...

Method of Vue component document generation tool library

Table of contents Parsing .vue files Extract docu...

How to directly reference vue and element-ui in html

The code looks like this: <!DOCTYPE html> &...

React non-parent-child component parameter passing example code

React is a JAVASCRIPT library for building user i...

Solve the compatibility issue between MySQL 8.0 driver and Alibaba Druid version

This article mainly introduces the solution to th...

4 flexible Scss compilation output styles

Many people have been told how to compile from th...

Implementation of docker-compose deployment project based on MySQL8

1. First, create the corresponding folder accordi...