Build Maven projects faster in Docker

I. Overview

This article will build docker images in the following ways, and record the build time of each method to get the fastest way to build Maven projects in Docker:

• Conventional multi-stage image building
• Building images with Buildkit
• Build images using layered dependencies
• Using volume mounts during Buildkit builds
• Building images using the Maven daemon

Between each run, we change the source code by adding a blank line; between each part, we delete all built images, including intermediate images as a result of multi-stage builds, in order to avoid reusing previously built images and to get more accurate build times for each method. Below is a simple spring boot project for testing.

2. Conventional multi-stage image building

Here is the relevant Dockerfile:

FROM openjdk:11-slim-buster as build                         

COPY .mvn .mvn                                               
COPY mvnw .                                                  
COPY pom.xml .                                               
COPY src src                                                 

RUN ./mvnw -B package                                        

FROM openjdk:11-jre-slim-buster                              

COPY --from=build target/fast-maven-builds-1.0.jar .         

EXPOSE 8080

ENTRYPOINT ["java", "-jar", "fast-maven-builds-1.0.jar"]

Let's execute the build:

time DOCKER_BUILDKIT=0 docker build -t fast-maven:1.0 .      

Forget about environment variables for now, I will explain in the next section

Here are the results from five runs:

* 0.36s user 0.53s system 0% cpu 1:53.06 total
* 0.36s user 0.56s system 0% cpu 1:52.50 total
* 0.35s user 0.55s system 0% cpu 1:56.92 total
* 0.36s user 0.56s system 0% cpu 2:04.55 total
* 0.38s user 0.61s system 0% cpu 2:04.68 total

3. Use Buildkit to build images

The previous command line uses the DOCKER_BUILDKIT environment variable, which is how you tell Docker to use the old engine. If you haven't updated Docker in a while, that's the engine you're using. Today, BuildKit has replaced it as the new default.

BuildKit brings several performance improvements:

• Automatic garbage collection
• Concurrent dependency resolution
• Efficient instruction cache
• Build cache import/export
• etc.

Let's re-execute the previous command on the new engine:

time docker build -t fast-maven:1.1 .

Here's an excerpt from the console log for the first run:

...
=> => transferring context: 4.35kB
=> [build 2/6] COPY .mvn .mvn
=> [build 3/6] COPY mvnw .
=> [build 4/6] COPY pom.xml .
=> [build 5/6] COPY src src
=> [build 6/6] RUN ./mvnw -B package
...

0.68s user 1.04s system 1% cpu 2:06.33 total

The following executions of the same command have slightly different output:

...
=> =>Transferring context: 1.82kB
=> CACHED [build 2/6] COPY .mvn .mvn
=> CACHED [build 3/6] COPY mvnw .
=> CACHED [build 4/6] COPY pom.xml .
=> [build 5/6] COPY src src
=> [build 6/6] RUN ./mvnw -B package
...

Remember that we changed the source code between runs. The files we don't change, namely .mvn, mvnw and pom.xml, are cached by BuildKit. But these resources are small, so caching won't improve build times significantly.

* 0.69s user 1.01s system 1% cpu 2:05.08 total
* 0.65s user 0.95s system 1% cpu 1:58.51 total
* 0.68s user 0.99s system 1% cpu 1:59.31 total
* 0.64s user 0.95s system 1% cpu 1:59.82 total

A quick look at the logs revealed that the biggest bottleneck in the build was the download of all dependencies (including plugins). This happens every time we make a source code change, which is why BuildKit doesn't improve performance.

4. Build images using dependency layering

We should focus on dependencies. To do this, we can make use of layers and split the build into two steps:

• The first step is to download dependencies
• In the second one, we do proper packaging

Each step creates a layer, and the second depends on the first.
By layering, if we change the source code in the second layer, the first layer is not affected and can be reused. We don't need to download dependencies again. The new Dockerfile looks like:

FROM openjdk:11-slim-buster as build

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .

RUN ./mvnw -B dependency:go-offline                          

COPY src src

RUN ./mvnw -B package                                        

FROM openjdk:11-jre-slim-buster

COPY --from=build target/fast-maven-builds-1.2.jar .

EXPOSE 8080

ENTRYPOINT ["java", "-jar", "fast-maven-builds-1.2.jar"]

NOTE: go-offline does not download everything. If you try to use the -o option (for offline), the command will not run successfully. This is a well-known old mistake. In all cases, it's "good enough".

Let's run the build:

time docker build -t fast-maven:1.2 .

The first run takes much longer than a regular build:

0.84s user 1.21s system 1% cpu 2:35.47 total

However, subsequent builds are much faster. Changing the source code affects only the second layer and does not trigger the download of (most) dependencies:

* 0.23s user 0.36s system 5% cpu 9.913 total
* 0.21s user 0.33s system 5% cpu 9.923 total
* 0.22s user 0.38s system 6% cpu 9.990 total
* 0.21s user 0.34s system 5% cpu 9.814 total
* 0.22s user 0.37s system 5% cpu 10.454 total

5. Using volume mounts during Buildkit builds

Layered builds greatly reduce build time, but there is still a problem. Changing a single dependency will invalidate the layer that the image depends on, so we need to download all dependencies again.

Fortunately, BuildKit introduces volume mounting during build time (not just during runtime). There are several types of mounts available, but the one we are interested in is the cache mount. This is an experimental feature, so you need to explicitly opt-in:

The Dockerfile looks like:

# syntax=docker/dockerfile:experimental                      
FROM openjdk:11-slim-buster as build

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .
COPY src src

# Use cache to build RUN --mount=type=cache,target=/root/.m2,rw ./mvnw -B package 

FROM openjdk:11-jre-slim-buster

COPY --from=build target/fast-maven-builds-1.3.jar .

EXPOSE 8080

ENTRYPOINT ["java", "-jar", "fast-maven-builds-1.3.jar"]

The # syntax=docker/dockerfile:experimental is used to enable experimental features.
Use the following command to build the image:

time docker build -t fast-maven:1.3 .

Build times are higher than regular builds, but still lower than layered builds:

0.71s user 1.01s system 1% cpu 1:50.50 total

The following constructs are equivalent to layers:

* 0.22s user 0.33s system 5% cpu 9.677 total
* 0.30s user 0.36s system 6% cpu 10.603 total
* 0.24s user 0.37s system 5% cpu 10.461 total
* 0.24s user 0.39s system 6% cpu 10.178 total
* 0.24s user 0.35s system 5% cpu 10.283 total

6. Building images using the Maven daemon

The Dockerfile file content for building an image using the Maven daemon is as follows:

FROM openjdk:11-slim-buster as build
# Download the latest version of Maven daemon ADD https://github.com/mvndaemon/mvnd/releases/download/0.6.0/mvnd-0.6.0-linux-amd64.zip . 
# Update package index RUN apt-get update \     
# Install unzip
 && apt-get install unzip \     
# Create a dedicated folder && mkdir /opt/mvnd \       
# Extract the mvnd file we downloaded earlier
 && unzip mvnd-0.6.0-linux-amd64.zip \ 
# Move the extracted archive contents to the folder created earlier && mv mvnd-0.6.0-linux-amd64/* /opt/mvnd                    

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .
COPY src src
# Use mvnd instead of Maven wrapper RUN /opt/mvnd/bin/mvnd -B package                            

FROM openjdk:11-jre-slim-buster

COPY --from=build target/fast-maven-builds-1.4.jar .

EXPOSE 8080

ENTRYPOINT ["java", "-jar", "fast-maven-builds-1.4.jar"]

Build the image using the following command:

time docker build -t fast-maven:1.4 .

The log output is as follows:

* 0.70s user 1.01s system 1% cpu 1:51.96 total
* 0.72s user 0.98s system 1% cpu 1:47.93 total
* 0.66s user 0.93s system 1% cpu 1:46.07 total
* 0.76s user 1.04s system 1% cpu 1:50.35 total
* 0.80s user 1.18s system 1% cpu 2:01.45 total

There is no significant improvement over regular build images.

VII. Conclusion

Here is a summary of all execution times:

Speeding up the performance of Maven builds in Docker is very different from regular builds, the limiting factor is the download speed of dependencies, and the use of layers is required to cache dependencies.
For BuildKit, it is recommended to use the new cache mount feature to avoid downloading all dependencies when a layer is invalidated.

References

https://blog.frankel.ch/faster-maven-builds/2/

This is the end of this article about building Maven projects faster in Docker. For more information about building Maven projects with Docker, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!