Spring Integration Tests with MongoDB Rulez

While unit testing is always preferable, integration tests are a good and necessary supplement to either perform end to end tests, or tests involving (third party) backends. Databases are such a candidate where integrations might make sense: usually we encapsulate persistence with some kind of repository service layer, which we can mock in tests running against the repository. But when it comes to testing the repository itself, integration tests are quite useful. Spring integration tests allow you to test functionality against a running Spring application, and thereby allows to test against a running database instance. But as you do in unit tests, you have to perform a proper set up of test data, and clean up the database afterwards. That's what this article is about: proper database set- and clean-up in Spring integration tests with MongoDB.

Tickets, Please

Let's first introduce our repository layer to test. Say we want to establish a simple ticket system which is hosted in a MongoDB. The ticket is quite simple: it has a description, a (non-technical) ticketId, and the MongoDB object id.

@Document
public class Ticket {

    @Id
    private ObjectId id;

    @Indexed(unique = true)
    private String ticketId;

    private String description;

Note: using a technical (MongoDB) ID as a functional ID is seldom a good idea, you should always keep that separate.
We want to enforce the uniqueness of ticket ID's by adding a unique index, which we can do in Spring by simply adding an @Indexed annotation.

The TicketRepository is also quite simple. Additionally to the standard CRUD methods it defines a method for retrieving a ticket by it's ID:

public interface TicketRepository extends MongoRepository {

    Ticket findByTicketId(final String ticketId);
}

As long as we do not provide custom persistence logic, you may argue, that is not necessary to write tests for standard Spring functionality. But here we implement business logic using database features - in this case the uniqueness of the ticket IDs - so at least this is worth writing a test. Let's begin with a simple test, that just creates two tickets, and looks 'em up using our finder method:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class TicketRepositoryIT {

    @Autowired
    private TicketRepository repository;
    @Autowired
    private MongoTemplate mongoTemplate;

    @Test
    public void testSaveAndFindTicket() throws Exception {
        Ticket ticket1 = new Ticket("1", "blabla");
        repository.save(ticket1);
        Ticket ticket2 = new Ticket("2", "hihi");
        repository.save(ticket2);

        assertEquals(ticket1, repository.findByTicketId("1"));
        assertEquals(ticket2, repository.findByTicketId("2"));
        assertNull(repository.findByTicketId("3"));
    }
}

Alright, let it run. Green, so what’s you point? Let it run again. Er, red? What the heck?!? The StackTrace is your friend:

org.springframework.dao.DuplicateKeyException: 

{ "serverUsed" : "localhost:27017" , "ok" : 1 , "n" : 0 , 

  "err" : "E11000 duplicate key error index: test.ticket.$ticketId dup key: { : \"1\" }" , "code" : 11000}; 

...

A DuplicateKeyException, eh? Makes sense: our tests always tries to create to tickets with IDs “1” and “2”. On our first attempt, the database was empty.  But on the second run, they already existed and the MongoDB complains about our unique index constraint being hurt.

Clean up your Test Rubbish

So we should nicely clean up our test dirt, before and after the test. Not that hard. Since test should always run against a dedicated database for testing purposes, we can simply drop the collection:

public class TicketRepositoryIT {
...
    @Before
    public void setup() throws Exception {
        mongoTemplate.dropCollection(Ticket.class);
    }

    @After
    public void tearDown() throws Exception {
        mongoTemplate.dropCollection(Ticket.class);
    }
...

Now run the test again. And again. Ah, green :-)

Spring and MongoDB Indices

Since we rely on the index to implement the business logic of unique ticket IDs, we should write a test to ensure this functionality. This is straight forward, we just create two tickets with the same ticket ID and expect a DuplicateKeyException)

    @Test(expected = DuplicateKeyException.class)
    public void testSaveNewTicketWithExistingTicketId() throws Exception {
        repository.save(new Ticket("1", "blabla"));
        repository.save(new Ticket("1", "hihi"));
    }

And if we run the test we get our expected...er, red?!? What happened? Well, Spring nicely creates all indices on the collection by inspecting our entity class on start up. But when we drop the collection, the indices are dropped also. Makes sense. But currently Spring does not re-create the indices on the next insert, see this issue for more details on the discussion. So in the meantime, we have to do this on ourselves. After picking up some knowledge from class MongoPersistentEntityIndexCreator we can easily write some code to recreate the index for a given entity class:

    protected void createIndecesFor(final Class<?> type) {
        final MongoMappingContext mappingContext =
                (MongoMappingContext) getMongoTemplate().getConverter().getMappingContext();
        final MongoPersistentEntityIndexResolver indexResolver =
                new MongoPersistentEntityIndexResolver(mappingContext);
        for (final IndexDefinitionHolder indexToCreate : indexResolver.resolveIndexForClass(type)) {
            createIndex(indexToCreate);
        }
    }

    private void createIndex(final IndexDefinitionHolder indexDefinition) {
        getMongoTemplate().getDb().getCollection(indexDefinition.getCollection())
                .createIndex(indexDefinition.getIndexKeys(), indexDefinition.getIndexOptions());
    }

All we have to do now, is to recreate the indices after dropping the collection in setup:

    @Before
    public void setup() throws Exception {
        mongoTemplate.dropCollection(Ticket.class);
        createIndecesFor(Ticket.class);
    }

    @After
    public void tearDown() throws Exception {
        mongoTemplate.dropCollection(Ticket.class);
    }

If we now rerun the tests... ah, green again :-D

Make it a Rule

Since we don’t want to copy that code into any, we could extract it into a base class, but… this really sucks. Junit 4 introduced rules to factor out such helper code. The ExternalResource rule is a quite perfect base for rules that are supposed to run before and after each test, so let’s do it that way:

public class MongoCleanupRule extends ExternalResource {
...
    @Override
    protected void before() throws Throwable {
        dropCollections();
        createIndeces();
    }

    @Override
    protected void after() {
        dropCollections();
    }

Since we want our rule to be configurable with one to n collections to clean up, we will pass the collection classes in the rule constructor:

    private final Class<?>[] collectionClasses;

    public MongoCleanupRule(final Class<?>... collectionClasses) {
        Assert.notNull(collectionClasses, "parameter 'collectionClasses' must not be null");
        Assert.noNullElements(collectionClasses,
                "array 'collectionClasses' must not contain null elements");

        this.collectionClasses = collectionClasses;
    }

    @Override
    protected void before() throws Throwable {
        dropCollections();
        createIndeces();
    }

    @Override
    protected void after() {
        dropCollections();
    }

    protected Class<?>[] getMongoCollectionClasses() {
        return collectionClasses;
    }

    protected void dropCollections() {
        for (final Class<?> type : getMongoCollectionClasses()) {
            getMongoTemplate().dropCollection(type);
        }
    }

    protected void createIndeces() {
        for (final Class<?> type : getMongoCollectionClasses()) {
            createIndecesFor(type);
        }
    }

This is straight forward: we just iterate over the given mongo collections, drop 'em, and recreate the indices for each one. But wait, where is the required MongoTemplate coming from?!?  Well, we could pass that in the constructor together with the collection classes. But if you remember our integration test, the template is injected by Spring using @Autowired, which is quite convenient. But we have no definite time, when the template gets injected, so we got to be lazy here. Instead of passing the template to the rule, the rule pulls it from the test class using reflection. The test class has to provide the template either in a member variable or a getter method, whose names are configurable. We define the default names to be mongoTemplate and getMongoTemplate.

public class MongoCleanupRule extends ExternalResource {

    private final Object testClassInstance;
    private final Class<?>[] collectionClasses;
    private final String fieldName;
    private final String getterName;

    public MongoCleanupRule(final Object testClassInstance, final Class<?>... collectionClasses) {
        this(testClassInstance, "mongoTemplate", "getMongoTemplate", collectionClasses);
    }

    public MongoCleanupRule(final Object testClassInstance, final String fieldOrGetterName,
            final Class<?>... collectionClasses) {
        this(testClassInstance, fieldOrGetterName, fieldOrGetterName, collectionClasses);
    }

    protected MongoCleanupRule(final Object testClassInstance, final String fieldName,
            final String getterName, final Class<?>... collectionClasses) {
        Assert.notNull(testClassInstance, "parameter 'testClassInstance' must not be null");
        Assert.notNull(fieldName, "parameter 'fieldName' must not be null");
        Assert.notNull(getterName, "parameter 'getterName' must not be null");
        Assert.notNull(collectionClasses, "parameter 'collectionClasses' must not be null");
        Assert.noNullElements(collectionClasses,
                "array 'collectionClasses' must not contain null elements");

        this.fieldName = fieldName;
        this.getterName = getterName;
        this.testClassInstance = testClassInstance;
        this.collectionClasses = collectionClasses;
    }
...
    protected MongoTemplate getMongoTemplate() {
        try {
            Object value = ReflectionTestUtils.getField(testClassInstance, fieldName);
            if (value instanceof MongoTemplate) {
                return (MongoTemplate) value;
            }
            value = ReflectionTestUtils.invokeGetterMethod(testClassInstance, getterName);
            if (value instanceof MongoTemplate) {
                return (MongoTemplate) value;
            }
        } catch (final IllegalArgumentException e) {
            // throw exception with dedicated message at the end
        }
        throw new IllegalArgumentException(
            String.format("%s expects either field '%s' or method '%s' in order to access the required MongoTemmplate",
                    this.getClass().getSimpleName(), fieldName, getterName));
    }
}

This reduces our integration test to the following lines:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class TicketRepositoryIT {

    @Autowired
    private TicketRepository repository;
    @Autowired
    private MongoTemplate mongoTemplate;

    @Rule
    public final MongoCleanupRule cleanupRule = new MongoCleanupRule(this, Ticket.class);

    @Test
    public void testSaveAndFindTicket() throws Exception {
        Ticket ticket1 = new Ticket("1", "blabla");
        repository.save(ticket1);
        Ticket ticket2 = new Ticket("2", "hihi");
        repository.save(ticket2);

        assertEquals(ticket1, repository.findByTicketId("1"));
        assertEquals(ticket2, repository.findByTicketId("2"));
        assertNull(repository.findByTicketId("3"));
    }

    @Test(expected = DuplicateKeyException.class)
    public void testSaveNewTicketWithExistingTicketId() throws Exception {
        repository.save(new Ticket("1", "blabla"));
        repository.save(new Ticket("1", "hihi"));
    }

Since our member variable holding the MongoTemplate has the default name defined in the rule, there is not much to set up in the rule. Just provide the test class instance itself (so we can access the template using reflection), and the collection class. That's all we need, just enough to get things done.

That's it. You can find this test project and the rule on GitHub.

Remember, rule number one: there are no rules!
Mick Jagger