Creating Test Stages With JUnit
Learn more about how to create test stages in JUnit 4 and 5.
Join the DZone community and get the full member experience.
Join For FreeOver the last few years, the prominent effectiveness of Continuous Integration (CI) and Continuous Delivery (CD) have cemented their places in the software distribution toolbox and are likely here to stay. One aspect of this move away from previous integration and delivery schemes is the reduction of the build, package, test, and delivery steps into stages that can be chained together into a pipeline. This allows for standalone portions of a build to be completed in serial or in parallel, with the completion of each step leading to the execution of subsequent steps.
This not only allows us to separate certain portions of a build from one another, but it also allows us to run more nimble parts of the build without waiting for more lengthy portions to complete. In particular, unit tests are often executed first, while integration, acceptance, performance, and stress tests are often executed later in a pipeline. This allows us to quickly know if the fast unit tests—which provide us a basic, yet essential coverage of isolated parts of our system—failed and stop all subsequent, long-running stages (i.e. fail-fast) which would otherwise waste time and resources.
In order to apply the efficiency and versatility of staged builds, we first need to break our build into stages that can be run independently of one another. In this article, we will look at how to break the execution of JUnit tests into categories, allowing us to execute unit and integration tests separately. This article includes information on how to execute these stages in both Maven and Gradle and will explore how to divide our tests into categories in both JUnit 4 and JUnit 5. All of the source code and build scripts for this article can be found in the following repositories:
Creating an Example Project
In order to demonstrate how to create test stages, we will first create a simple example project that includes objects that we can test in isolation using unit tests, as well as more complex objects, such as a Representation State Transfer (REST) controller, that will require integration tests to fully cover. This distinction between unit and integration tests will eventually lead to the development of two test stages that can be executed independently of one another.
Note that this project uses Spring 5 WebFlux to create the REST controller, but it is not required that the reader fully understand Spring 5 or even the mechanism used to test a Spring 5 controller. Instead, any desired framework can be used, but conceptually, it should be understood that we are creating a REST controller that requires more than simple unit tests to exercise. The details of the REST controller and its associated integration tests will be explained at a high-level, but the interested reader should refer to the Guide to Spring 5 WebFlux and the Spring 5 WebClient and WebTestClient Tutorial with Examples.
Our project will follow the standard design of creating domain objects (one in our case: Foo
), creating a repository object to manage the persistence of our domain objects (FooRepository
), and creating a REST controller to allow clients to access our domain objects over Hypertext Transfer Protocol (HTTP) (FooController
). Since this is a simple project, we will create an in-memory repository object, rather than a repository backed by persistent storage, such as a database or file system.
Being that we only need to store an ID for our domain objects, our Foo
domain class ends up being trivially simple:
public class Foo {
private long id;
public Foo() {}
public Foo(long id) {
this.id = id;
}
public void setId(long id) {
this.id = id;
}
public long getId() {
return id;
}
}
With our domain class established, we can now implement our repository class:
@Service
public class FooRepository {
private final List<Foo> foos = new ArrayList<>();
public void save(Foo foo) {
foos.add(foo);
}
public Optional<Foo> findById(long id) {
return foos.stream()
.filter(foo -> foo.getId() == id)
.findFirst();
}
public List<Foo> findAll() {
return foos;
}
}
The @Service
annotation informs the Spring framework that this class is a candidate to be autowired into another object using the Spring Dependency Injection (DI) framework. This will be important when we create our REST controller later. This class stores its Foo
objects in a List
and has three methods that allow a client to add a new Foo
object to the repository, find an existing Foo
object by its ID, or obtain a List
of all existing Foo
objects.
Lastly, with our domain and repository classes implemented, we create our REST controller:
@RestController
@RequestMapping("/foo")
public class FooController {
@Autowired
private FooRepository repository;
@GetMapping("/{id}")
public Mono<Foo> findById(@PathVariable Long id) {
return repository.findById(id)
.map(Mono::just)
.orElseThrow();
}
@ResponseStatus(HttpStatus.NOT_FOUND)
@ExceptionHandler(NoSuchElementException.class)
public void handleNotFound() {}
}
The @RestController
annotation informs the Spring framework that this controller should be included as a part of the HTTP server that is started by Spring when the application is executed. The @RequestMapping
annotation informs Spring that we wish for this controller to have the path /foo
(i.e. http://localhost:8080/foo
). The @Autowired
annotation instructs Spring to inject an object of the FooRepository
class that we previously created, allowing us to access Foo
domain objects within our controller. The findById
method accepts the desired ID as a path variable and to find a Foo
object in the FooRepository
with a matching ID. If one is found, it is converted to a Mono
object and returned or else a NoSuchElementException
is thrown (using the orElseThrow
method).
The handleNotFound
method is responsible for handling this exception (using the @ExceptionHandler
annotation to inform Spring of its ability to handle a NoSuchElementException
) and converting it to an HTTP 404 response (as denoted by the @ResponseStatus
annotation). While the three classes we have created are relatively simple, they require some nuance when deciding which level of testing is appropriate.
Adding Unit and Integration Tests
Since our domain object is trivially simple, and for the sake of conciseness, we will forgo adding JUnit tests to exercise its functionality. Instead, we will create a set of unit tests for our FooRepository
to ensure that it functions as intended. Since our FooController
class has interconnected dependencies with our FooRepository
and the Spring WebFlux framework, we will create a set of integration tests to exercise its functionality.
Two important points to note: (1) although there is no syntactic difference between a JUnit unit test and a JUnit integration test (JUnit simply provides a mechanism for executing automated tests, regardless of the nature of the tests themselves), we will make a logical separation between unit and integration tests using annotations; and (2) both unit and integration tests should be created prior to the implementation of the classes they exercise, in line with the practices of Test-Driven Development (TDD). For the purposes of brevity, we did not walk through the process of creating a test and then implementing a small portion of code to get that test to pass. Instead, the tests in the following section can be thought of as the final result after the test suite and desired classes were fully implemented. In practice, though, these tests would be created using TDD.
JUnit 4
In order to differentiate the unit tests from integration tests, we will use JUnit 4 Categories. In the case of our unit tests, we will not define a category and instead treat them as the default set of tests that will be executed whenever a build is run (unless explicitly restricted from executing). This results in the following unit test class for our FooRepository
:
public class FooRepositoryTest {
private FooRepository repository;
@Before
public void setUp() {
repository = new FooRepository();
}
@Test
public void noExistingFoosResultsInFooNotFoundById() {
assertTrue(repository.findAll().isEmpty());
assertFalse(repository.findById(1).isPresent());
}
@Test
public void existingFooNotMatchingIdResultsInFooNotFoundById() {
long desiredId = 1;
long differentId = 2;
repository.save(new Foo(differentId));
assertFalse(repository.findById(desiredId).isPresent());
}
@Test
public void existingFooMatchingIdResultsInFooFoundById() {
Foo foo = new Foo(1);
repository.save(foo);
Optional<Foo> result = repository.findById(foo.getId());
assertTrue(result.isPresent());
assertEquals(foo, result.get());
}
}
We are simply exercising three cases in which a desired Foo
object may be found by its ID: (1) no Foo
objects exists, which results in no Foo
object found by any ID (i.e. return an empty Optional
), (2) a Foo
object with a different ID exists, which results in no Foo
object being found by the desired ID, and (3) there exists a Foo
object with the desired ID, which results in the desired Foo
object being returned (i.e. an Optional
populated with the desired Foo
object).
For the integration test that exercises our FooController
, we need to create a marker interface that is used by the JUnit @Category
annotation to categorize our test class. We will aptly name this interface IntegrationTest
:
public interface IntegrationTest {}
We will also need to create a Spring configuration that informs Spring how it should inject our desired dependencies and where possible injection candidates should be found:
@Configuration
@ComponentScan("com.dzone.albanoj2.example.junitstages")
public class ControllerTestConfig {}
In this case, the @Configuration
annotation informs Spring that this class should be used a configuration class and the @ComponentScan
annotation states that Spring should look for injection candidates in the com.dzone.albanoj2.example.junitstages
package or any of its sub-packages. Note that this package name will change based on the package structure of a particular project. Using this configuration, we can define our test cases for FooController
:
@WebFluxTest
@RunWith(SpringRunner.class)
@Category(IntegrationTest.class)
@ContextConfiguration(classes = ControllerTestConfig.class)
public class FooControllerTest {
@Autowired
private WebTestClient webClient;
@MockBean
private FooRepository repository;
@Test
public void findByIdWithNoFoosResultsIn404() {
webClient.get()
.uri("/foo/{id}", 1)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isNotFound();
}
@Test
public void findByIdWithFooWithDifferentIdResultsInFooNotFound() {
long desiredId = 1;
long differentId = 2;
Foo foo = new Foo(differentId);
doReturn(Optional.of(foo)).when(repository).findById(eq(differentId));
webClient.get()
.uri("/foo/{id}", desiredId)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isNotFound();
}
@Test
public void findByIdWithMatchingFoosResultsInFooFound() {
long id = 1;
Foo foo = new Foo(id);
doReturn(Optional.of(foo)).when(repository).findById(eq(id));
webClient.get()
.uri("/foo/{id}", id)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody(Foo.class);
}
}
The @WebFluxTest
configures the test class to execute using the default Spring WebFlux configuration (i.e. how the test class should be bootstrapped, configuring the WebTestClient
, etc.), the @RunsWith
annotation instructs JUnit to execute this test class using the SpringRunner
test runner, the @Category
annotation categorizes the test cases using the marker interface we previously created, and the @ContextConfiguration
annotation instructs Spring to use the ControllerTestConfig
class we previously created.
The @MockBean
annotation allows for a mock FooRepository
object to be injected into both the test class and the FooController
class. This allows us to define the expected outputs for the FooRepository
class when running our test cases, which in turn allows us to control the output received by the FooController
when calling methods on the FooRepository
that was autowired into it. Each of the tests cases simply executes an HTTP GET request using the injected WebTestClient
object and inspects the results. Note that the doReturn
calls are Mockito methods that configure the mock FooRepository
object to return a certain value when that method is called (see the Mockito class documentation for more information).
JUnit 5
Creating the equivalent test classes using JUnit 5 is similar to the steps laid out in the previous section, but there are some important differences. Chiefly among these differences that the @Category
annotation has been replaced with the @Tag
annotation. The @Tag
annotation allows test classes to be categorized using a String
value rather than a marker interface, but that introduces another problem. For each of the test classes (suppose we had 10 integration test classes), we would need to repeat the exact same String
value when tagging each class (i.e. we would need to annotate every integration test class with @Tag("integration")
and hope that we did not mis-type "integration"
or define some constant and ensure we used the same constant everywhere). Note that any String
value can be used (i.e. integration
does not represent any special value in JUnit).
Fortunately, JUnit 5 can process meta-annotations, which allows us to create an @IntegrationTest
annotation that can be used in place of @Tag("integration")
:
@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Tag("integration")
public @interface IntegrationTest {}
This means that when JUnit 5 sees a test class annotated with @IntegrationTest
, it processes that class as if it were annotated with @Tag("integration")
instead. With this annotation defined, we can create our test classes. First, we define the FooRepository
unit test class in the same way we did for JUnit 4, replacing JUnit 4 annotations with JUnit 5 annotations:
public class FooRepositoryTest {
private FooRepository repository;
@BeforeEach
public void setUp() {
repository = new FooRepository();
}
@Test
public void noExistingFoosResultsInFooNotFoundById() {
assertTrue(repository.findAll().isEmpty());
assertFalse(repository.findById(1).isPresent());
}
@Test
public void existingFooNotMatchingIdResultsInFooNotFoundById() {
long desiredId = 1;
long differentId = 2;
repository.save(new Foo(differentId));
assertFalse(repository.findById(desiredId).isPresent());
}
@Test
public void existingFooMatchingIdResultsInFooFoundById() {
Foo foo = new Foo(1);
repository.save(foo);
Optional<Foo> result = repository.findById(foo.getId());
assertTrue(result.isPresent());
assertEquals(foo, result.get());
}
}
Next, we define our Spring configuration class in exactly the same manner as JUnit 4:
@Configuration
@ComponentScan("com.dzone.albanoj2.example.junitstages")
public class ControllerTestConfig {}
Lastly, we define the integration test class for our FooController
in a slightly different manner:
@WebFluxTest
@IntegrationTest
@ContextConfiguration(classes = ControllerTestConfig.class)
public class FooControllerTest {
@Autowired
private WebTestClient webClient;
@MockBean
private FooRepository repository;
@Test
public void findByIdWithNoFoosResultsIn404() {
webClient.get()
.uri("/foo/{id}", 1)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isNotFound();
}
@Test
public void findByIdWithFooWithDifferentIdResultsInFooNotFound() {
long desiredId = 1;
long differentId = 2;
Foo foo = new Foo(differentId);
doReturn(Optional.of(foo)).when(repository).findById(eq(differentId));
webClient.get()
.uri("/foo/{id}", desiredId)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isNotFound();
}
@Test
public void findByIdWithMatchingFoosResultsInFooFound() {
long id = 1;
Foo foo = new Foo(id);
doReturn(Optional.of(foo)).when(repository).findById(eq(id));
webClient.get()
.uri("/foo/{id}", id)
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody(Foo.class);
}
}
A majority of the test class is identical to the JUnit example, but there are two important differences to note: (1) we annotate our test class with @IntegrationTest
rather than @Category
, and (2) we can remove the @RunWith
annotation, as the default-configured runner supplied by JUnit 5 is sufficient. Apart from these two differences, the rest of the test code itself remains unchanged. Regardless of which JUnit version is used, we are now ready to configure our build scripts to execute only certain tests, depending on the build command executed.
Configuring the Build Scripts
In recent years, both Maven and Gradle have been the main players in the Java build ecosystem. While there are a number of similarities between Maven and Gradle, the process of executing test cases based on their annotations is very different. The following sections will walk through the process of configuring either Maven or Gradle to execute unit and integration tests separately in either JUnit 4 or JUnit 5.
Maven
With Maven, we can separate the execution of unit and integration tests using build profiles. Build profiles allow us to change the configuration of a Maven build depending on the selected profile. In our case, we will configure Maven to execute our unit tests whenever a build is performed with the default profile (i.e. no build profile is supplied) and execute only our integration tests (no unit tests) when the integration build profile is supplied. Note that the build profile can be named any valid profile name (not just integration
) and that integration
does not represent any special value in Maven.
Using the build profile mechanism, by default, we will configure the maven-surefire-plugin
, which executes our JUnit tests, to exclude any integration tests, denoted by the <excludedGroups>
element. If the integration build profile is selected, we will override this default maven-surefire-plugin
configuration and include only integration tests (utilizing the @Category
or @Tag
annotations, depending on the JUnit version used) using the <groups>
element. By specifying a value for <groups>
, all other groups will be excluded (see Using JUnit Categories).
The specifics of how to do this vary by the JUnit version we use.
JUnit 4
Using JUnit 4, we have to exclude or include groups using the fully-qualified class name of the class supplied to @Category
. In our case, that means com.dzone.albanoj2.example.junitstages.rest.IntegrationTest
. Simply add the following build plugin to the project pom.xml
file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>com.dzone.albanoj2.example.junitstages.rest.IntegrationTest</excludedGroups>
<argLine>--illegal-access=permit</argLine>
</configuration>
</plugin>
The <excludedGroups>
element instructs Surefire to run any tests that do not have a category of com.dzone.albanoj2.example.junitstages.rest.IntegrationTest
. Note that this has been verified to work with Surefire plugin version 2.1.1.RELEASE
and mileage may vary with older versions. Also note that the <argLine>--illegal-access=permit</argLine>
is needed when compiling with Java 9 or higher and is not specific to having different test stages (i.e. it can be safely removed if compiling with Java 8 or lower).
Next, add the following build profile to the pom.xml
file:
<profile>
<id>integration</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<groups>com.dzone.albanoj2.example.junitstages.rest.IntegrationTest</groups>
<excludedGroups>none</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
Note that <id>
element corresponds to the name of our build profile. The <groups>
element instructs Surefire to include only those test classes that have a category of com.dzone.albanoj2.example.junitstages.rest.IntegrationTest
(all others will be excluded). We include a dummy value (none
) for <excludedGroups>
to overwrite the <excludedGroups>com.dzone.albanoj2.example.junitstages.rest.IntegrationTest</excludedGroups>
element that specified as the default Surefire plugin configuration. Note that if <excludedGroups>
is not explicitly supplied in our build profile, the <excludedGroups>
from our Surefire plugin configuration will be used instead, which would both exclude and include the same group.
Note that none
is not a special value, but rather, a category class name that is unlikely to ever be used. If, hypothetically, there was a class with a fully-qualified class name of none
on the classpath for our project, we could run into trouble.
JUnit 5
The configuration and thought process is identical for JUnit 5, but instead of using the fully-qualified class name, we use the String value we provided to the @Tag
annotation (i.e. integration
). For the Surefire plugin, we add the following to our pom.xml
file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludedGroups>integration</excludedGroups>
<argLine>--illegal-access=permit</argLine>
</configuration>
</plugin>
For the build profile, we add the following to the pom.xml
file:
<profile>
<id>integration</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<groups>integration</groups>
<excludedGroups>none</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
Note that the profile ID and the group name do not have to match, but in this case, they do.
Running the Test Stages
Regardless of the JUnit version used, the unit tests can be executed using the following command (note that the clean
phase can be excluded if a clean build is not needed):
mvn clean test
Likewise, the integration tests can be executed by specifying the integration
build profile using the following command (again, clean
is optional):
mvn clean test -P integration
Note that any phase that includes the test phase will also execute either the unit tests or integration tests (if a build profile is supplied). For example, mvn clean install
will execute the unit tests while mvn clean install -P integration
will likewise execute the integration tests.
Gradle
Although the thought process is similar for Gradle, the mechanism we use is different. Instead of using build profiles and overriding the default Surefire configuration, we define two tasks: One for unit tests and another for integration tests.
JUnit 4
In the case of JUnit 4, we will define the following tasks in the build.gradle
file:
test {
useJUnit {
excludeCategories 'com.dzone.albanoj2.example.junitstages.rest.IntegrationTest'
}
testLogging {
events 'PASSED', 'FAILED', 'SKIPPED'
}
}
task integrationTest(type: Test) {
useJUnit {
includeCategories 'com.dzone.albanoj2.example.junitstages.rest.IntegrationTest'
}
check.dependsOn it
shouldRunAfter test
testLogging {
events 'PASSED', 'FAILED', 'SKIPPED'
}
}
In both the test
and integrationTest
tasks (we override the test
task already defined by Gradle), we simply specify which categories to include or exclude (includeCateories
and excludeCategories
, respectively) using the fully-qualified class name of the class we supplied to the @Category
annotation (see this Stack Overflow post). We also include testLogging
so that we can see the results of each of the tests that run, but this is not required (and the test case output is omitted by default if no testLogging
element is supplied). In this case, we configure both tasks to print test logging output for the PASSED
, FAILED
, and SKIPPED
events.
Note that in the integrationTest
task, we also create an ordering for the Gradle build steps. Using the check.dependsOn it
statement, we tell Gradle that the integrationTest
task should be run whenever the check
task is run. Similarly, using the shouldRunAfter test
statement, we also instruct Gradle to execute the integrationTest
task after the test
task if both tasks are run in the same Gradle build.
JUnit 5
The build configuration for JUnit 5 is nearly identical, but instead of fully-qualified class names, we use the String
-based tag names instead (i.e. integration
) and replace includeCategories and excludeCategories
with includeTags
and excludeTags
, respectively:
test {
useJUnitPlatform {
excludeTags 'integration'
}
testLogging {
events 'PASSED', 'FAILED', 'SKIPPED'
}
}
task integrationTest(type: Test) {
useJUnitPlatform {
includeTags 'integration'
}
check.dependsOn it
shouldRunAfter test
testLogging {
events 'PASSED', 'FAILED', 'SKIPPED'
}
}
Running the Test Stags
Unlike our Maven case, which overrides the default Surefire plugin configuration, the unit and integration test tasks are independent tasks, which allows them to be run separately or both in the same build. To run the unit tests only, execute the following command (the clean
task can be removed if a clean build is not needed):
gradle clean test
To execute the integration tests only, execute the following command (again, clean
is optional):
gradle clean integrationTest
To run both the unit and integration tests, execute the following command (clean
is optional):
gradle clean test integrationTest
Note that if another task includes the test
or integrationTest
tasks, the unit or integration tests, respectively, will run. For example, running the following command will also execute both the unit and integration tests:
gradle clean check
Conclusion
With the advent of CI and CD, building software in stages has become the norm among many of the most successful software products. Staged, independent build steps, and particularly staged tests, allow for pipelines to fail-fast without wasting time and resources executing long-running tests on a build that has already failed. Not only does this introduce independence and flexibility in the build process, but it also allows for pipelined stages to be executed in parallel, dramatically reducing build times and providing engineering and operations with more fine-grained control over how builds are executed.
Opinions expressed by DZone contributors are their own.
Comments