DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Related

  • Maven Plugin Testing - in a Modern Way - Part I
  • Creating Test Stages With JUnit
  • The Most Popular Technologies for Java Microservices Right Now
  • Micronaut With Relation Database and...Tests

Trending

  • Faster Startup With Spring Boot 3.2 and CRaC, Part 2
  • Cost Optimization Strategies for Managing Large-Scale Open-Source Databases
  • MuleSoft: Best Practices on Batch Processing
  • Front-End Debugging, Part 1: Not Just Console Log
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. JUnit Test Groups for More Reliable Development

JUnit Test Groups for More Reliable Development

By 
Horatiu Dan user avatar
Horatiu Dan
·
Nov. 19, 20 · Tutorial
Likes (4)
Comment
Save
Tweet
Share
7.4K Views

Join the DZone community and get the full member experience.

Join For Free

Introduction

As a product is being developed and maintained its test suite is enriched as features and functionalities are added and enhanced. Ideally, development teams should aim having a lot of quick unit tests that are run whenever modifications in the code are made. It is great if these are written before or at least together with the tested code, cover as many use cases as possible and finish running after a reasonable amount of time. By all means, a reasonable amount of time is an entity difficult to quantify. 

On the other hand, there are a lot of live products that solve quite complex business problems whose thorough verification concerns cross-cutting through multiple application layers at once and thus, the number of integration tests is significant. Running all these unit and integration tests whenever the code is touched is not always feasible, as the productivity and the development speed is decreased considerably. Again, productivity and development speed are hard to quantify but should be always traded for correctness. 

Under such circumstances, developers need to imagine possible compromises that help delivering confidently and reliably. One such compromise is the ability to split quick and small unit tests from the heavier integration tests. 

This post is going to present a small working example that describes how to split tests into multiple categories (groups) and accomplish the above mentioned compromise.

Context

‘testcategories’ is a small project especially developed for the purpose of this post, using the following: 

  • Java 11
  • Apache Maven 3.6.3
  • JUnit 4.13
  • Maven Surefire Plugin 2.22.0

It contains two classes each simulating the retrieval of a certain category of entities available from two different sources. They are great sports people, either Romanian or international.

Java 
xxxxxxxxxx
1
21
 
1 
public class Local {
2 
    
3 
    public List<String> champions() {
4 
        System.out.println("Retrieving local champions...");
5 
        return List.of("Simona Halep", "Cristina Neagu");
6 
    }
7 
}
8
9 
public class Remote {
10
11 
    public List<String> champions() {
12 
        try {
13 
            System.out.println("Retrieving remote champions...");
14 
            Thread.sleep(5000);
15 
            
16 
            return List.of("Roger Federer", "Dan Carter");
17 
        } catch (InterruptedException e) {
18 
            throw new RuntimeException("Champions unavailable.");
19 
        }
20 
    }
21 
}


For the sake of this example, it is imagined that the ‘local’ ones are easy to reach, cheaper to retrieve in terms of the involved resources, while the ‘remote’ ones more expensive. This is simulated by forcing the designated method to last at least 5 seconds, by putting the current thread to sleep. 

Under these circumstances, it is reasonable to state that Local#champions() method could be unit tested, while the Remote#champions() one integration tested. 

Solutions

In order to split unit and integration tests into distinct categories, there are at least two approaches. Both may be accomplished with the help of ‘maven-surefire-plugin’ by configuring it accordingly. 

  • Adopt a naming convention for the test classes, depending on the test category they belong to and configure Maven Surefire Plugin so that it includes / excludes what is executed in each scenario. 
XML 
xxxxxxxxxx
1
12
 
1 
<plugin>
2 
    <artifactId>maven-surefire-plugin</artifactId>
3 
    <version>2.22.0</version>
4 
    <configuration>
5 
        <includes>
6 
            <include>**/*Test.java</include>
7 
        </includes>
8 
        <excludes>            
9 
            <exclude>**/*Integration.java</exclude>
10 
        </excludes>
11 
    </configuration>
12 
</plugin>


It is expected that classes whose names end in ‘Test’ will designate unit tests, while those suffixed with ‘Integration’, integration tests. 

I have experimented this solution before, it’s been working great for years, the only drawback I find is that one cannot have both unit and integration tests residing in the same class. 

  • Annotate classes or tests inside a class with JUnit’s @Category and categorize all or a single test as being part of a certain group designated by a custom marker interface. Moreover, configure Maven Surefire Plugin to run the desired group(s) of tests.
XML 
xxxxxxxxxx
1
 
1 
<plugin>
2 
    <groupId>org.apache.maven.plugins</groupId>
3 
    <artifactId>maven-surefire-plugin</artifactId>
4 
    <version>2.22.0</version>
5 
    <configuration>                      
6 
        <groups>com.hcd.testcategories.config.UnitTests</groups>
7 
    </configuration>
8 
</plugin>


As part of this post, the latter solution is detailed. 

Split Unit and Integration Test

The sample project contains 5 tests that are organized in 4 different classe so that multiple scenarios can be presented. In order to split them, two categories are defined as two marker interfaces, whose names were chosen so that it is clear what they designate. 

Java 
xxxxxxxxxx
1
 
1 
public interface UnitTests {
2
3 
}
4
5 
public interface IntegrationTests {
6
7 
}


In order for a class to be considered part of a group, it is enough to annotate it with @Category and provide as value the corresponding marker interface – @Category(UnitTests.class). 

Moreover, one can put a test into multiple groups by providing the value as comma separated marker interfaces – @Category({SlowTests.class, DatabaseTests.class}). 

Nevertheless, in the example illustrated here, it is a nonsense to consider a test both as unit and integration in the same time. 

The mentioned tests are detailed below. 

  • LocalTest – contains 1 unit test – class is annotated with @Category(UnitTests.class) 
Java 
xxxxxxxxxx
1
12
 
1 
@Category(UnitTests.class)
2 
public class LocalTest {
3
4 
    @Test  
5 
    public void champions() {
6 
        final Local local = new Local();        
7 
        List<String> result = local.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Simona Halep"));
11 
    }
12 
}


  • RemoteTest – contains 1 integration test – class is annotated with @Category(IntegrationTests.class)
Java
  




xxxxxxxxxx
1
12


 
1 
@Category(IntegrationTests.class)
2 
public class RemoteTest {
3
4 
    @Test
5 
    public void champions() {
6 
        final Remote remote = new Remote();    
7 
        List<String> result = remote.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Roger Federer"));
11 
    }
12 
}



  • MixedTest – contains 1 unit and 1 integration test – class is not annotated, but each test is annotated with @Category(UnitTests.class) and @Category(IntegrationTests.class) respectively 
Java 
xxxxxxxxxx
1
22
 
1 
public class MixedTest {
2
3 
    @Test
4 
    @Category(UnitTests.class)
5 
    public void localChampions() {
6 
        final Local local = new Local();        
7 
        List<String> result = local.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Simona Halep"));
11 
    }
12 
    
13 
    @Test
14 
    @Category(IntegrationTests.class)
15 
    public void remoteChampions() {
16 
        final Remote remote = new Remote();    
17 
        List<String> result = remote.champions();
18 
        
19 
        assertEquals(2, result.size());
20 
        assertTrue(result.contains("Roger Federer"));
21 
    }
22 
}


  • NewLocalTest – contains 1 unit test – neither the class, nor the test is annotated
Java 
xxxxxxxxxx
1
11
 
1 
public class NewLocalTest {
2
3 
    @Test  
4 
    public void champions() {
5 
        final Local local = new Local();        
6 
        List<String> result = local.champions();
7 
        
8 
        assertEquals(2, result.size());
9 
        assertTrue(result.contains("Cristina Neagu"));
10 
    }
11 
}


The intention with the last one is to designate a test for which the developer has forgotten to annotate it and thus, accidentally left it out of any of the two categories (groups). Such a situation is very likely to appear in real life, after the point the existing tests have been split. 

Maven Configuration

In order to be easier to run the tests in various scenarios, one can define a Maven property that designates the test categories (groups) taken into account at a certain execution. Moreover, a couple of profiles can be used, so that it is easier to invoke the build / test command. 

Below is the complete POM file of the sample project. 

XML 
xxxxxxxxxx
1
86
 
1 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
2 
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3
4 
    <modelVersion>4.0.0</modelVersion>
5
6 
    <groupId>com.hcd</groupId>
7 
    <artifactId>testcategories</artifactId>
8 
    <version>0.0.1-SNAPSHOT</version>
9 
    <description>Sample Maven Project that differentiate between Unit and Integration tests at runtime.</description>
10 
  
11 
    <properties>      
12 
        <test.categories/>
13 
    </properties>
14 
  
15 
    <profiles>
16 
        <profile>
17 
            <id>all-tests</id>          
18 
            <activation>
19 
                <activeByDefault>true</activeByDefault>
20 
            </activation>
21 
            <properties>
22 
                <build.profile.id>all-tests</build.profile.id>
23 
                <test.categories/>
24 
            </properties>
25 
        </profile>
26 
        <profile>
27 
            <id>unit-tests</id>        
28 
            <properties>
29 
                <build.profile.id>unit-tests</build.profile.id>
30 
                <test.categories>com.hcd.testcategories.config.UnitTests</test.categories>
31 
            </properties>
32 
        </profile>
33 
        <profile>
34 
            <id>integration-tests</id>
35 
            <properties>
36 
                <build.profile.id>integration-tests</build.profile.id>
37 
                <test.categories>com.hcd.testcategories.config.IntegrationTests</test.categories>
38 
            </properties>
39 
        </profile>
40 
    </profiles>
41
42 
    <build>
43 
        <pluginManagement>
44 
            <plugins>
45 
                <plugin>
46 
                    <groupId>org.apache.maven.plugins</groupId>
47 
                    <artifactId>maven-compiler-plugin</artifactId>  
48 
                    <version>3.8.0</version>
49 
                    <configuration>
50 
                        <release>11</release>
51 
                        <includeEmptyDirs>true</includeEmptyDirs>
52 
                    </configuration>                  
53 
                </plugin>            
54 
                <plugin>
55 
                    <groupId>org.apache.maven.plugins</groupId>
56 
                    <artifactId>maven-surefire-plugin</artifactId>
57 
                    <version>2.22.0</version>
58 
                    <configuration>                      
59 
                        <groups>${test.categories}</groups>
60 
                    </configuration>
61 
                </plugin>
62 
            </plugins>
63 
        </pluginManagement>
64 
        
65 
        <plugins>
66 
            <plugin>
67 
                <groupId>org.apache.maven.plugins</groupId>
68 
                <artifactId>maven-compiler-plugin</artifactId>
69 
            </plugin>
70 
            <plugin>
71 
                <groupId>org.apache.maven.plugins</groupId>
72 
                <artifactId>maven-surefire-plugin</artifactId>
73 
            </plugin>
74 
        </plugins>
75 
    </build>
76 
        
77 
    <dependencies>
78 
        <dependency>
79 
        <groupId>junit</groupId>
80 
            <artifactId>junit</artifactId>
81 
            <version>4.13</version>
82 
            <scope>test</scope>
83 
        </dependency>
84 
    </dependencies>
85 
</project>
85 
</project>


The profiles defined are: 

  • ‘all-tests’ – default profile, executes all unit, integration and non-categorized tests. test.categories property is empty.
  • ‘unit-tests’ – executes all annotated unit tests. test.categories = com.hcd.testcategories.config.UnitTests.
  • ‘integration-tests’ – executes all annotated integration tests. test.categories = com.hcd.testcategories.config.IntegrationTests.

Running Scenarios

  • Run all defined tests – unit, integration and non-categorized

  • by invoking the default profile implicitly

> mvn clean test 

  • by invoking the ‘all-tests’ profile 

> mvn clean test -Pall-tests 

  • by specifying an empty ‘test.categories’ property 

> mvn clean test -Dtest.categories=

All 5 tests are executed. The output is below. 

PowerShell 
xxxxxxxxxx
1
27
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.107 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving remote champions...
9 
Retrieving local champions...
10 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.959 s - in com.hcd.testcategories.MixedTest
11 
[INFO] Running com.hcd.testcategories.NewLocalTest
12 
Retrieving local champions...
13 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 s - in com.hcd.testcategories.NewLocalTest
14 
[INFO] Running com.hcd.testcategories.RemoteTest
15 
Retrieving remote champions...
16 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.964 s - in com.hcd.testcategories.RemoteTest
17 
[INFO]
18 
[INFO] Results:
19 
[INFO]
20 
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
21 
[INFO]
22 
[INFO] ------------------------------------------------------------------------
23 
[INFO] BUILD SUCCESS
24 
[INFO] ------------------------------------------------------------------------
25 
[INFO] Total time:  14.651 s
26 
[INFO] Finished at: 2020-11-12T12:21:31+02:00
27 
[INFO] ------------------------------------------------------------------------


  •  Run all annotated tests – unit and integration

  • by specifying the groups (comma separated) as values for the ‘test.categories’ property

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.UnitTests,com.hcd.testcategories.config.IntegrationTests

All 4 annotated tests are executed. The non-annotated one in NewLocalTest is not. The output is below. 

PowerShell 
xxxxxxxxxx
1
25
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving remote champions...
9 
Retrieving local champions...
10 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.942 s - in com.hcd.testcategories.MixedTest
11 
[INFO] Running com.hcd.testcategories.RemoteTest
12 
Retrieving remote champions...
13 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.986 s - in com.hcd.testcategories.RemoteTest
14 
[INFO]
15 
[INFO] Results:
16 
[INFO]
17 
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
18 
[INFO]
19 
[INFO] ------------------------------------------------------------------------
20 
[INFO] BUILD SUCCESS
21 
[INFO] ------------------------------------------------------------------------
22 
[INFO] Total time:  14.580 s
23 
[INFO] Finished at: 2020-11-12T13:08:21+02:00
24 
[INFO] ------------------------------------------------------------------------


  • Run all annotated unit tests

  • by invoking the ‘unit-tests’ profile

> mvn clean test -Punit-tests

  • by specifying the unit tests group as a value for the ‘test.categories’ property 

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.UnitTests 

Both tests that are in the custom UnitTests category are executed – one from the LocalTest annotated class and one from the non-anotated MixedTest class, but annotated at method level. The output is below. 

PowerShell 
xxxxxxxxxx
1
21
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving local champions...
9 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 s - in com.hcd.testcategories.MixedTest
10 
[INFO]
11 
[INFO] Results:
12 
[INFO]
13 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
14 
[INFO]
15 
[INFO] ------------------------------------------------------------------------
16 
[INFO] BUILD SUCCESS
17 
[INFO] ------------------------------------------------------------------------
18 
[INFO] Total time:  5.891 s
19 
[INFO] Finished at: 2020-11-12T13:27:19+02:00
20 
[INFO] ------------------------------------------------------------------------


  • Run all annotated integration tests

  • by invoking the ‘integration-tests’ profile

> mvn clean test -Pintegration-tests 

  • by specifying the integration tests group as a value for the ‘test.categories’ property

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.IntegrationTests 

Both tests that are in the custom IntegrationTests category are executed – one from the RemoteTest annotated class and one from the non-anotated MixedTest class, but annotated at method level. The output is below. 

PowerShell
x
21
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.MixedTest
5 
Retrieving remote champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.993 s - in com.hcd.testcategories.MixedTest
7 
[INFO] Running com.hcd.testcategories.RemoteTest
8 
Retrieving remote champions...
9 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.967 s - in com.hcd.testcategories.RemoteTest
10 
[INFO]
11 
[INFO] Results:
12 
[INFO]
13 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
14 
[INFO]
15 
[INFO] ------------------------------------------------------------------------
16 
[INFO] BUILD SUCCESS
17 
[INFO] ------------------------------------------------------------------------
18 
[INFO] Total time:  15.140 s
19 
[INFO] Finished at: 2020-11-12T13:46:22+02:00
20 
[INFO] ------------------------------------------------------------------------


Conclusions

  • The quickest execution is, without any doubt, the one that runs just the explicitly marked unit tests.
  • The most thorough execution is the one that runs all defined tests (irrespective of the defined groups / categories); it definitely takes longer, but it is the most reliable one in terms of confidence.
  • No written test will cease to run during the default execution if for example a developer forgets to annotate either the containing test class or the test method (see point 1. and NewLocalTest class in the previous section).
  • One should apply each of the presented scenarios, depending on the particular circumstance and the purpose of the aimed achievement. During development the former seems to be the proper candidate, while the latter is more suitable for pre-production builds.

Code

The sample project is available now in GitHub - testcategories

unit test Apache Maven Integration JUnit integration test

Published at DZone with permission of Horatiu Dan. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Maven Plugin Testing - in a Modern Way - Part I
  • Creating Test Stages With JUnit
  • The Most Popular Technologies for Java Microservices Right Now
  • Micronaut With Relation Database and...Tests

Partner Resources

Comments

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: