Grouping Tests Using JUnit Categories

In a well-organized build process, you want lightning-fast unit tests to run first, and provide whatever feedback they can very quickly. A nice way to do this is to be able to class your tests into different categories. For example, this can make it easier to distinguish between faster running unit tests, and slower tests such as integration, performance, load or acceptance tests. This feature exists in TestNG, but, until recently, not in JUnit.

Indeed, this has been missing from the JUnit world for a long time. Using JUnit, I typically use test names (integration tests end in 'IntegrationTest', for example) or packages to identify different types of test. It is easy to configure a build script using Maven or Ant to run different types of test at different points in the build lifecycle. However it would be nice to be able to do this in a more elegant manner.

JUnit 4.8 introduced a new feature along these lines, called Categories. However, like most new JUnit features, it is almost entirely undocumented. In this article we'll see how it works and what it can do for you.

In JUnit 4.8, you can define your own categories for your tests. Categories are implemented as classes or interfaces. Since they simply act as markers, I prefer to use interfaces. One such category interface might look like this:

public interface IntegrationTests {}

You can also use inheritance to organize your test categories:

public interface SlowTests {}
public interface IntegrationTests extends SlowTests {}
public interface PerformanceTests extends SlowTests {}

So far so good. Now you can use these categories in your tests. In this example we flag a particular test class as containing integration tests:

@Category(IntegrationTests.class)
public class AccountIntegrationTest {
   
    @Test
    public void thisTestWillTakeSomeTime() {
        ...
    }
    
    @Test
    public void thisTestWillTakeEvenLonger() {
        ....
    }
}

You can also flag individual test methods if you prefer:

public class AccountTest {
   
    @Test
	@Category(IntegrationTests.class)
    public void thisTestWillTakeSomeTime() {
        ...
    }
    
    @Test
	@Category(IntegrationTests.class)
    public void thisTestWillTakeEvenLonger() {
        ...
    }

    @Test
    public void thisOneIsRealFast() {
        ...
    }
}

To run tests in a particular category, you need to set up a test suite. In JUnit 4, a test suite is essentially an empty annotated class. To run only tests in a particular category, you use the @Runwith(Categories.class) annotation, and specify what category you want to run using the @IncludeCategory annotation

@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@SuiteClasses( { AccountTest.class, ClientTest.class })
public class LongRunningTestSuite {}

You can also ask JUnit not to run tests in a particular category using the @ExcludeCategory annotation

@RunWith(Categories.class)
@ExcludeCategory(SlowTests.class)
@SuiteClasses( { AccountTest.class, ClientTest.class })
public class UnitTestSuite {}

Test categories are great if you use JUnit test suites. I haven't used test suites for years: Maven can find all my tests by itself, thank you very much, so I don't have to remember to add my test classes to the right test suite each time a create a new one. However, test suites do give you finer control over what order your tests are executed in, so you might still find them useful in that regard.

Once you've done this, it is then easy to run tests in a particular category from within your IDE simply by running the test suite.

On the tooling and build automation side of things, JUnit categories are not supported as well as TestNG groups. For example, the Maven surefire plugin lets you specify the TestNG groups you want to run in a particular phase, but no such support exists as yet for JUnit categories. You can of course configure the Surefire plugin to run a particular test suite (or test suites) in a particular phase, but it doesn't dispense you with the need to write and maintain a test suite.

So test categories are great, but having to run them via a test suite (and to remember to add new test classes to the test suite) seems a bit clunky in these days of annotations and reflection.

From http://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0