MySQL can actually implement distributed locks

Preface

In the previous article, I shared with you how to use locks in a monolithic architecture through the example of flash sales in e-commerce scenarios. However, many application systems are now quite large, and many application systems are microservice architecture systems. So in this cross-JVM scenario, how should we solve concurrency?

Limitations of Monolithic Application Locks

Before we get into the actual practice, let me briefly talk with you about the architectural evolution in Internet systems.

At the beginning of the development of the Internet system, the resource consumption was relatively small and the number of users was relatively small. We only needed to deploy one tomcat application to meet the needs. We can regard a tomcat as a jvm process. When a large number of requests arrive at the system concurrently, all requests fall on this only tomcat. If some request methods need to be locked, such as the scenario of flash sale and inventory reduction mentioned in the previous article, it can meet the needs. However, as the number of visits increases, one Tomcat cannot support it. At this time, we need to deploy Tomcat in a cluster and use multiple Tomcats to support the system.

After the simple evolution in the above figure, we deploy two Tomcats to jointly support the system. When a request arrives at the system, it will first pass through nginx, which will act as a load balancer and forward the request to one of the tomcats according to its own load balancing configuration strategy. When a large number of requests are accessed concurrently, the two tomcats share all the traffic. After this, when we conduct flash sales to reduce inventory, can we still meet the demand by using a single application lock?

The lock we added before is the lock provided by JDK. This lock works under a single JVM. When there are two or more, a large number of concurrent requests are scattered to different Tomcats. Concurrency can be prevented in each Tomcat. However, between multiple Tomcats, the request to obtain the lock in each Tomcat generates concurrency again. Therefore, the problem of deducting inventory still exists. This is the limitation of single application lock. So how do we solve this problem? Next, I will share with you the distributed lock.

Distributed Locks

What is a distributed lock?

So what is a distributed lock? Before talking about distributed locks, we see that the characteristic of a single application lock is that it is effective in one JVM, but cannot cross JVMs and processes. So we can give a less official definition: a distributed lock is a lock that can span multiple JVMs and multiple processes. A lock like this is a distributed lock.

Design ideas

Since Tomcat is started by Java, each Tomcat can be regarded as a JVM, and the lock inside the JVM cannot span multiple processes. Therefore, when we implement distributed locks, we can only look for them outside these JVMs and implement them through other components.

In the above figure, the two Tomcats use third-party components to implement cross-jvm and cross-process distributed locks. This is the solution to distributed locks.

Implementation

So what third-party components are currently available to achieve this? The more popular ones are the following:

• Databases can implement distributed locks through databases, but high concurrency puts a lot of pressure on the database, so they are rarely used.
• Redis, with the help of redis, distributed locks can be implemented, and there are many types of redis java clients, so the usage methods are also different.
• Zookeeper can also implement distributed locks. Similarly, zk also has many java clients, and the usage methods are also different.

For the above implementation methods, Lao Mao still demonstrates them one by one through specific code examples.

Distributed lock based on database

Idea: Implement distributed locks based on database pessimistic locks, mainly using select ... for update. The purpose of select ... for update is to lock the queried data during the query. When the user performs this kind of operation, other threads are prohibited from modifying or deleting the data. They must wait until the previous thread completes the operation and releases it before they can proceed, thus achieving the effect of locking.

Implementation: We will share the code with you based on the example of overselling in e-commerce.

Let's use the overselling example in the last monolithic architecture to share with you. We will modify the last code and create a new table called distribute_lock. The main purpose of this table is to provide database locks. Let's take a look at the situation of this table.

Since we are simulating an oversold order scenario, we have a lock data for an order in the above figure.

We will modify the code in the previous article to extract a controller and then request the call through postman. Of course, two JVMs are started in the background to operate, namely port 8080 and port 8081. The completed code is as follows:

/**
 * @author [email protected]
 * @date 2021/1/3 10:48
 * @desc Public account "Programmer Old Cat"
 */
@Service
@Slf4j
public class MySQLOrderService {
  @Resource
  private KdOrderMapper orderMapper;
  @Resource
  private KdOrderItemMapper orderItemMapper;
  @Resource
  private KdProductMapper productMapper;
  @Resource
  private DistributeLockMapper distributeLockMapper;
  //Purchase product id
  private int purchaseProductId = 100100;
  //Quantity of purchased goods private int purchaseProductNum = 1;
  
  @Transactional(propagation = Propagation.REQUIRED)
  public Integer createOrder() throws Exception{
    log.info("Entered the method");
    DistributeLock lock = distributeLockMapper.selectDistributeLock("order");
    if(lock == null) throw new Exception("The distributed lock for this business is not configured");
    log.info("got the lock");
    //Here we will sleep for 1 minute to demonstrate the concurrency manually. Thread.sleep(60000);

    KdProduct product = productMapper.selectByPrimaryKey(purchaseProductId);
    if (product==null){
      throw new Exception("Purchase product: "+purchaseProductId+" does not exist");
    }
    //Current inventory of the product Integer currentCount = product.getCount();
    log.info(Thread.currentThread().getName()+"inventory count"+currentCount);
    //Check inventory if (purchaseProductNum > currentCount) {
      throw new Exception("Product"+purchaseProductId+" has only "+currentCount+" items left, cannot be purchased");
    }

    //Complete the reduction operation in the database productMapper.updateProductCount(purchaseProductNum,"kd",new Date(),product.getId());
    //Generate order... The number of times is omitted. The source code can be downloaded from Lao Mao's github: https://github.com/maoba/kd-distribute
    return order.getId();
  }
}

The SQL is written as follows:

select
  *
  from distribute_lock
  where business_code = #{business_code,jdbcType=VARCHAR}
  for update

The above is the main implementation logic. Note the following points in the code:

• The createOrder method must have a transaction because the select for update lock can only be triggered when a transaction exists.
• The code must determine the existence of the current lock. If it is empty, an exception will be reported.

Let's take a look at the final running effect. First, take a look at the console log.

The console log of 8080 is as follows:

11:49:41 INFO 16360 --- [nio-8080-exec-2] ckdservice.MySQLOrderService : Entered method
11:49:41 INFO 16360 --- [nio-8080-exec-2] ckdservice.MySQLOrderService : Got the lock

The console log of 8081 is as follows:

11:49:48 INFO 17640 --- [nio-8081-exec-2] ckdservice.MySQLOrderService : Entered method

From the log, we can see that the first request to 8080 of two different JVMs gets the lock first, so the request to 8081 is waiting for the lock to be released before it can be executed. This shows that our distributed lock is effective.
Let's take a look at the log after the complete execution:

Request for 8080:

11:58:01 INFO 15380 --- [nio-8080-exec-1] ckdservice.MySQLOrderService : Entered method
11:58:01 INFO 15380 --- [nio-8080-exec-1] ckdservice.MySQLOrderService : Got the lock
11:58:07 INFO 15380 --- [nio-8080-exec-1] ckdservice.MySQLOrderService : http-nio-8080-exec-1 Inventory count 1

Request for 8081:

11:58:03 INFO 16276 --- [nio-8081-exec-1] ckdservice.MySQLOrderService : Entered method
11:58:08 INFO 16276 --- [nio-8081-exec-1] ckdservice.MySQLOrderService : Got the lock
11:58:14 INFO 16276 --- [nio-8081-exec-1] ckdservice.MySQLOrderService : http-nio-8081-exec-1 Inventory count 0
11:58:14 ERROR 16276 --- [nio-8081-exec-1] oaccC[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.Exception: Only 0 items left for product 100100, cannot be purchased] with root cause

java.lang.Exception: There are only 0 items left for item 100100, and it cannot be purchased.
at com.kd.distribute.service.MySQLOrderService.createOrder(MySQLOrderService.java:61) ~[classes/:na]

Obviously, the second request failed due to lack of inventory. Of course, this scenario is also in line with our normal business scenario. Finally, our database looks like this:

Obviously, the inventory and order quantities we have in this database are also correct. At this point, our practical demonstration of distributed locks based on the database is complete. Let's summarize the advantages and disadvantages of using this type of lock.

• Advantages: simple and convenient, easy to understand and easy to operate.
• Disadvantages: When the concurrency is large, the pressure on the database will be greater.
• Recommendation: Separate the lock database from the business database.

Final Thoughts

As for the above-mentioned database distributed locks, they are actually rarely used in our daily development. Redis and ZK-based locks are more commonly used. Originally, I wanted to share Redis locks and ZK locks together in this article, but if they were written in the same article, the length would be too long, so this article will share this type of distributed lock with you. You can download the source code from Lao Mao's GitHub. The address is: https://github.com/maoba/kd-distribute

This is the end of this article about how MySQL can actually implement distributed locks. For more relevant MySQL distributed lock content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!