Optimize the Execution Time of Spring Integration Tests

Introduction

The article is roughly divided into four parts:

1. The first part will briefly introduce some real project data, including time-consuming data before and after optimization so that friends can understand how much optimization space is there for integration testing, and highlight why integration testing needs to be optimized.
2. The second part will introduce the composition of integration test execution time through some examples so that everyone knows where the time is spent.
3. In the third part, I will remove the "cloud" of the Spring test framework source code, and take the friends to briefly understand the principles of Spring integration testing.
4. Finally, I will give some useful suggestions for optimizing the execution time of Spring integration tests.

Integration Testing

Here are a few words. We have been pursuing automated testing for a long time. The fundamental reason is that automated testing is more reliable and cheaper than humans. Integration testing is an integral part of automated testing.

Doing a good job of integration testing in the team can bring confidence to the team. For example,

• The integration test covers a larger range, and it can cover some places that the unit test cover cannot.
• It can be more confident and boldly refactored.

In order to do a good job of integration testing, we also need to pay some price such as:

• Slow execution of integration tests (relative to unit tests).
• Construct test data to "cry"...

Why Do You Need to Reduce Test Execution Time?

Due to work reasons, only two real code repositories are posted below. Here, Gradle is used to generate the execution report of each Gradle task, and the execution report is reversed by time for each task.

The data are all in the picture. Friends can take a look first and ask themselves if this time-consuming is acceptable.

• 001 code base: 650+ total tests, including unit tests and integration tests
• 002 code base: 1250+ total tests, including unit tests and integration tests

Maybe some friends will think, is this is it? I poured a glass of water. It was just right when I came back from the bathroom last time. It may not be enough for this time-consuming...

However, as a Developer who is constantly exploring the limits, this time-consuming is unacceptable. If there is room for optimization, you should do it.

Next, I will optimize the 001 codebases in the first stage, where P0 refers to the time-consuming data before optimization.

P1 has a relatively general reduction in time-consuming:

• Test time: from 6'34s (P0) to 5'31s (P1), a total of 1'03s less.
• Total time: from 8'16s (P0) to 6'48s (P1), a total of 1'28s less.

But, it was still not enough and, then, I carried out the second stage of optimization.

P2 has a significant reduction in time-consuming:

• Test time: from 6'34s (P0) to 2'53s (P2), a total of 3'41s less.
• Total time: from 8'16s (P0) to 4'3s (P2), a total of 4'13s less.

In fact, I used very little energy to optimize, but I got very good results. Can my friends still accept the time-consuming P0 phase?

In actual projects, there are many integration tests that are not standardized, and the time-consuming can be much longer than the time-consuming of the P0 phase here. This will lead to untimely test feedback and slow CI efficiency.

At this point, the friends should understand how much room for optimization is in integration testing. This is why we need to optimize integration testing.

Where Is the Time Spent on Integration Testing?

Next, we move on to the second part; where did the time-consuming testing phase go?

This problem is actually very simple. We only need to get the test report output by Gradle build. This test report has counted the time consumption of each test file. For example, I intercepted part of the test report of the 001 codebases.

Duration refers to the time consumed by the test itself, and it does not include the time to prepare, run, and destroy the spring context. So, the total time for the test seems to be very small (866 tests include unit tests and integration tests).

We enter an integration test file and select Standard output. You may see two types of logs:

The first type of log obviously starts the Spring context, as shown below:

The second type of log has no Spring context, as shown below:

Up until this point, the question of where the time is spent has been roughly concluded. Here, I will summarize the time consumption in two parts:

1. The test itself (the test code we wrote).
2. Building the environment required for testing. For example, prepare, start, and destroy the Spring container, start and destroy the in-memory database, etc.

Among them, the test itself takes very little time (unless sleep or large disk IO operations are used), and the environment required to build the test is very time-consuming. The impact of this time-consuming is related to the size and complexity of the code base. You can run several integration tests on your own project to get the time-consuming data, which usually takes more than 10s.

Imagine if there are many integration tests that need to restart the context and how time-consuming it would be.

What Is the Principle of Spring Integration Testing?

Now that we know where the time-consuming integration test is spent, how should we optimize it?

In fact, when I clearly said how to optimize, there will still be various doubts among my friends. I will simply take you to look at the source code and understand the principles of Spring integration testing. Once I figure out the principles, I naturally know how to optimize. It is better to teach people how to fish.

Due to the limited space and the main focus of the article is optimization, I ignored a lot of details in the third part and only show the main context of the integration test. As for the principle of finer granularity, I will consider writing another article. Partners can also look at the source code by themselves.

The author uses JUnit5 and Spring Boot 2.4.5. Here, I assume that my friends have a certain knowledge of JUnit5.

As you can see from the above figure, JUnit5 Test Engine is responsible for executing tests, Spring Test Framework is responsible for building the environment required for integration testing, and we only need to pay attention to what the Spring Test Framework is doing.

Build TestContext

After we start the integration test, the Spring framework will first build a Test Context, the entry is @SpringBootTest . The corresponding source code is as follows:

We focus only @BootstrapWith(SpringBootTestContextBootstrapper.class)on the line, although the @ExtendWith(SpringExtension.class)principle of understanding Spring integration testing of some help, but not the focus of this article, for the time being, ignored.

SpringBootTestContextBootstrapperProvides many methods, among which we are most concerned about the buildTestContext()Build Test Context step in the flowchart. This method provides a test context and ApplicationContextprovides input for building Spring later.

Find/Build Application Context

Here is the most core code!

Here is the most core code!

Here is the most core code!

Prepare Spring TestContext, and then TestContextbuild Spring based on Spring to ApplicationContextprepare for the subsequent startup of the Spring Application. This operation is in DefaultTestContext.getApplicationContext(), the source code is as follows:

As can be seen from the code, here will be taken from the cache first ApplicationContext, and then we can enter the following:

cacheAwareContextLoaderDelegate.loadContext(this.mergedContextConfiguration);

Then, we look at the implementation logic of the cache, the source code corresponds to DefaultCacheAwareContextLoaderDelegate.loadContext(mergedContextConfiguration)

In which, context cache, made a synchronization lock in order to avoid loadContext(mergedContextConfiguration)method executed multiple times, resulting in repeated start Spring Application.

Looking at it, we can temporarily draw two conclusions:

1. Spring will take priority from the cache ApplicationContext
2. If the cache is not found, it will build a new one ApplicationContextand put it in the cache.

Then, we will have two doubts:

1. MergedContextConfiguration What is it?
2. What factors will affect the ContextCachefetch ApplicationContext?

For the first question, we can open the MergedContextConfiguration.javadescription:

We can see from the description when we perform a test class, Spring will use the integration test configurations are combined, placed MergedContextConfigurationin administration. Then, use it as a key to ApplicationContextthe cache ContextCahce.

What specific configurations will be merged? Friends can look at the description and the implementation code of this class. The implementation code is in AbstractTestContextBootstrapper.buildMergedContextConfiguration()

For the second question, we can look at MergedContextConfiguration.hashCode()

You can see the impact from ContextCachethe take ApplicationContextquite multifactorial, probably have the following:

1. Use @ContextConfiguration to customize different configurations between integration test classes.
2. Use different @ActiveProfiles configurations between integration test classes.
3. Use different @TestPropertySource configurations between integration test classes.
4. Does the integration test class inherit from the parent class?
5. Whether different custom contexts are used between the integration test classes is mainly maintained Set<ContextCustomizer>, and friends can see by themselves.

I focus Lecture 5:00, ContextCustomizerone implementation class MockitoContextCustomizer, it refers to @MockBeanand @SpyBeantwo, also integrates two Annotation tests are often used. If the two tests use different classes @Mockbeanand @SpyBeanwill lead ApplicationContextcan not be reused in the two test classes, leading to rebuilding ApplicationContext because hashCode () is different !!!

So, do not mess with @Mockbeanand @SpyBean!!!

So, do not mess with @Mockbeanand @SpyBean!!!

So, do not mess with @Mockbeanand @SpyBean!!!

Run Spring Application

After the above is built ApplicationContext, we will start the Spring application and make the final preparations for building the environment required for integration testing.

Code SpringBootContextLoader.loadContext(mergedContextConfiguration)and SpringApplication.run(args).

Since it’s not the focus of this article, I won’t bother to talk about it. Interested friends can just check it out.

Store Application Context

Cache Spring is ApplicationContextalready mentioned above to find/build Spring Application Context, so I won’t repeat it here.

After so much longing, the third part is also finished, which can be roughly summarized into three points:

1. Each class start integration testing, taking into account the efficiency, Spring will preferentially from ContextCachefetchApplicationContext
2. If the ContextCachefail to the ApplicationContextwill build a new ApplicationContext, then start Spring Application, and ApplicationContextcachedContextCache
3. When you start each class integration testing, Spring will use the integration test configurations are combined, put MergedContextConfigurationin management. Then, it will be as a key ApplicationContextcache ContextCahce, and once configured, each of the integrated test classes is different. You can not re-use ApplicationContext, resulting in the need to reload the entire Spring context.

How to Reduce the Time-Consuming Integration Test?

To clearly understand the general principles of Spring integration testing, I believe that my friends have a basic optimization strategy (the core goal is to reuse the ApplicationContext to effectively reduce the execution time).

Establish Norms

We first need to reach some consensus within the team; what is the scope of integration testing?

The teams may not be the same. Let me sort it out according to personal empiricism. After decomposing, there are roughly four consensuses:

• Which ones need to be integrated testing?

Code that does not involve external dependencies should be considered in the category of integration testing.

For example, Controller, Service, DB (in-memory database can be used instead)

• Which ones do not require integration testing?

Those external dependencies that make testing unstable should not be included in the scope of integration testing.

For example, MQ and FeignClient

• How to isolate code that does not require integration testing?

Use @MockBean

• How do you organize a custom configuration for integration testing?

Give priority to global reuse, try to make as little special as possible for the test class, and then unify it into a common base class for management (included @MockBean), and directly inherit the base class when writing integration tests.

E.g:

Refactor @MockBean

Use @MockBean with caution!!!

@MockBeanis often misused in integration testing, which is one of the main reasons for slow integration testing. Many people that use @MockBeanwrite a bunch of meaningless tests for convenience and use them extensively. It also makes integration tests very slow.

Unless external dependencies, the best way is not to be lazy. The data should be created. Most of the time, integration testing is spent on creating data. If it is not easy to create, consider other methods. For example, you can declare @SpyBeanit is Move to the base class, then build a technical debt card and arrange a time to refactor the code.

Sink Unit Test

You can consider sinking some integration tests into unit tests.

Unit testing does not need to prepare as much context as integration testing, so the execution time is very short.

Put It Into Practice

Put it into practice and, then, draw a conclusion in practice. If you have any questions, you can leave a message.

To Sum-Up

In this article, we first highlight the importance of optimizing integration testing through some data points, then we analyze the time-consuming components, take the friends to sort out the principles of Spring integration testing, and an in-depth understanding leads to Spring the root cause of the slow integration test. Finally, some suggestions for optimizing the integration test are given, hoping to be useful.

You are welcome to follow my WeChat subscription account. I will continue to output more technical articles, and I hope we can learn from each other.