Home Java Maven build acceleration

Maven build acceleration

by admin

Learn how to make Maven builds faster and more efficient

Builds require several properties, the most important of which is reproducibility.

I believe that speed should be lower in order of priority.However, it is also one of the most limiting factors on your release cycle :if your assembly takes T , you can’t release faster than each of the T

Consequently, you will probably want to speed up your builds after reaching a certain level of maturity in order to produce more frequent releases.

In this post, I want to detail some methods you can use to speed up Maven builds.The next post will focus on how to do the same inside Docker.

Source level

Since I want to propose methods and evaluate their impact, we need a sample repository.

I’ve selected Hazelcast code examples, because it provides a fairly large multi-module codebase with many submodules; the exact commit is 448febd

The procedure is as follows :

  • I run the command five times to avoid temporary problems

  • I perform mvnclean before each startup to start with an empty target repository

  • All dependencies and plugins already loaded

  • I report the time that Maven displays in the console log :

    [INFO] -------------------------------------------------------[INFO] BUILD SUCCESS[INFO] -------------------------------------------------------[INFO] Total time:22.456 s (Wall Clock)[INFO] Finished at: 2021-09-24T23:20:41+02:00[INFO] -------------------------------------------------------

Let’s start with our baseline by running the command : mvn test The result is :

  • 02:00 min.

  • 01:57 min.

  • 01:58 min.

  • 01:56 min.

  • 01:58 min.

Usage of all processors

By default, Maven uses a single thread.In the age of multi-core processors, this is a waste of time.You can run parallel assemblies using multiple threads by specifying an absolute number or a number relative to the number of available cores. For more information, please see related documentation

The more submodules you have that are independent of each other, i.e. Maven can build them in parallel, the better result you will get with this technique.

This works very well on our code base.

We are going to use as many threads as there are cores available. The corresponding Maven command: mvn test-T 1C

When you run this command you should see the following message in the console :

Using the MultiThreadedBuilder implementation with a thread count of X
  • 51.487 s (Wall Clock)

  • 40.322 s (Wall Clock)

  • 52.468 s (Wall Clock)

  • 41.862 s (Wall Clock)

  • 41.699 s (Wall Clock)

The numbers are much better, but with more variance.

Parallel execution of the test

Parallelization is a great method. We can do the same for test execution. By default, the Maven Surefire plugin runs tests sequentially, but you can configure it to run tests in parallel. Please refer to documentation For a complete set of options.

This approach is great if you have a large number of units in each module. Note that your tests should be independent of each other.

We manually set the number of threads :

mvn test -Dparallel=all -DperCoreThreadCount=false -DthreadCount=16 #1 #2
  1. Configure Surefire to run classes and methods in parallel

  2. Manually change the number of threads up to 16

Run and get :

  • 02:04 min.

  • 02:03 min.

  • 01:46 min

  • 01:52 min

  • 01:53 min.

It seems that the cost of thread synchronization offsets the potential benefit of running parallel tests.

Offline

Maven will check to see if the SNAPSHOT dependency a new "version" at each startup. This means extra network requests. We can prevent this check with the option --offline

Although you should avoid SNAPSHOT dependencies, sometimes it is unavoidable, especially during development.

Command mvn test -o , -o is shorthand for --offline

  • 01:46 min

  • 01:46 min

  • 01:47 min

  • 01:55 min.

  • 01:44 min

The code base has a large number of SNAPSHOT dependencies; hence, offline mode greatly speeds up the build.

JVMparameters

Maven itself is a Java-based application. This means that each run launches a new JVM The JVM first interprets the bytecode, a then analyzes the working code and compiles the bytecode in native code : this provides maximum performance, but only after a long build time. This is great for long running processes and not for command line applications.

We probably won’t get maximum performance in the context of assemblies because they are relatively short-lived, but we still pay the price of analysis time.

We can configure Maven to drop it by configuring the appropriate JVM settings. There are several ways to configure the JVM. The easiest way is to create a special jvm.config configuration file in the subfolder mvn projects.

-XX: -TieredCompilation -XX: TieredStopAtLevel = 1

Now let’s just run mvn test :

  • 01:44 min

  • 01:44 min

  • 01:53 min.

  • 01:53 min.

  • 01:55 min.

Maven Daemon

Demon Maven is a recent addition to the Maven ecosystem. It is inspired by the by the demon Gradle :

Gradle runs on a Java Virtual Machine (JVM) and uses several support libraries that require significant initialization time.

As a result, this can sometimes cause startup to be a bit slow. The solution to this problem is Gradle Daemon: a long-term background process that runs your builds much faster than without it. This is achieved by eliminating the costly initial boot process and using caching to keep your project data in memory.

The Gradle team realized that the command line tool is not the best use of the JVM. To solve this problem, the JVM’s background process has to be constantly maintained using a daemon. It runs as a server, and command line interface plays the role of a client.

As an added advantage, this long process loads classes only once (if they have not changed between runs).

After installing the software, you can start the daemon with mvnd command instead of the standard mvn Here are the results of the command : mvnd test :

  • 33.124 s (Wall Clock)

  • 33.114 s (Wall Clock)

  • 34.440 s (Wall Clock)

  • 32.025 s (Wall Clock)

  • 29.364 s (Wall Clock)

Note that by default the daemon uses multiple threads with the extension number of cores -1

Combining methods

We’ve seen several ways to speed up assembly. What if we used them together?

Let’s first try using all the methods we’ve seen so far in the same run :

mvnd test -Dparallel=all -DperCoreThreadCount=false -DthreadCount=16 -o #1 #2 #3 #4
  1. Using the Maven daemon

  2. Running the tests in parallel

  3. We do not update SNAPSHOT dependencies

  4. Configure the JVM settings as above, through jvm.config file – it is not necessary to set any parameters

The command returns the following results :

  • 27.061 s (Wall Clock)

  • 24.457 s (Wall Clock)

  • 24.853 s (Wall Clock)

  • 25.772 s (Wall Clock)

Let’s remember that the demon Maven is a long process. For this reason, it is reasonable to let the JVM analyze and compile bytecode in native code Thus, we can remove the jvm.config file and re-run the above command. Results :

  • 23.840 s (Wall Clock)

  • 26.589 s (Wall Clock)

  • 22.283 s (Wall Clock)

  • 23.788 s (Wall Clock)

  • 22.456 s (Wall Clock)

We can now display summary results :

Baseline

Parallel Build

Parallel Tests

Offline

Jvm Params

Daemon

Daemon + Offline + Parallel Tests+ Parameters

Daemon + Offline + Parallel Tests

#1 (s)

120.00

51.00

128.00

106.00

104.00

33.12

27.06

23.84

#2 (s)

117.00

40.00

123.00

106.00

104.00

33.11

24.46

26.59

#3 (s)

118.00

52.00

106.00

107.00

113.00

34.44

24.85

22.28

#4 (s)

116.00

42.00

112.00

115.00

113.00

32.03

25.77

23.79

#5 (s)

118.00

42.00

113.00

104.00

115.00

29.36

22.46

Average (s)

*117.80*

45.40

116.40

107.60

109.80

32.41

25.54

23.79

Deviation

1.76

25.44

63.44

14.64

22.96

2.91

1.00

2.38

Gain from baseline (s)

72.40

1.40

10.20

8.00

85.39

92.26

94.01..

% gain

61.46%

1.19%

8.66%

6.79%

72.48%

78.32%

79.80%

Conclusion

In this post, we’ve seen several ways to speed up Maven builds. Here is a summary of them :

  • Demon Maven : a reliable and safe starting point

  • Parallelizing an assembly : When an assembly contains several modules that are independent of each other.

  • Parallelization of tests : when a project contains multiple tests.

  • Autonomy : when a project contains SNAPSHOT dependencies and you do not need to update them.

  • JVM parameters : when you want to do your best

I would advise everyone to start using the Maven daemon and continue optimizing as needed and depending on your project.

In the next post, we’ll focus on speeding up your Maven builds in a container.

To take it further, read :

You may also like