The Anatomy of Good Unit Testing

I think of unit tests as an extension to my code. A thorough testing process offers peace of mind that when refactoring the code or making performance improvements, the units still function as expected. It can also find bugs and edge cases and avoid regressions during refactoring.

I come from a.NET / C# background and have compiled this collection of thoughts and tidbits that I find useful when writing tests.

Why Write Unit Tests?

I find that writing unit tests and writing robust maintainable code often go hand in hand. If the logic you are testing is mixed in with UI components, you may find that the components need to be loaded (possibly in a UI thread) in order for the tests to have access to the logic. If many classes are tightly coupled together, then the setup for a unit test may end up being convoluted. These and many other examples of unit testing becoming difficult is a sign that the code could benefit from some good refactoring.

Approaches to Unit Testing

Black box testing is where you test a unit without having access to the units source code, or by refusing to look at it. To do black box testing, you either know, get told, or experiment with what is expected of the unit.

White box testing is where you examine the source code of the unit under test to aid in writing the unit tests. By examining the source code, you’ll be able to make sure that your tests collectively cover all of the code paths within each action. This includes catering for the bounds of any loops, and following all logical conditions. It is certainly possible to achieve this level of code coverage with black box testing too, such as if the unit is simple enough to try out all the possible inputs or state combinations. White box testing can help capture obscure scenarios that you might not think about without seeing the code.

Testing state involves performing an action on a unit and then checking that either an expected result was returned, or the state of the unit has been updated as expected. Testing state can be achieved regardless of if you are white box or black box testing.

Testing implementation is an extension to white box testing where you check that certain methods are invoked or not during the execution of an action. Here you don’t assert any state, but instead, verify that internal behavior is doing what is expected.

The Anatomy of a Unit Test

A single unit test that asserts state is typically made up of the following three stages:

1. ‘Arrange’ gets everything ready to perform the test. This could be declaring variables, building required objects or setting the state of a unit based on the circumstances we want to test. Some or all of this step could take place in the SetUp method of the current unit testing fixture. For simple unit tests, this step may not be needed.
2. ‘Act’ performs the action that we are testing on the unit.
3. ‘Assert’ checks to see that the action performed correctly. We want to check that either the return value of a method call is expected, or the state of an object is as expected.

An example being:

[Test]
public void GetMinimum_UnsortedIntegerArray_ReturnsSmallestValue() 
{   
  var unsortedArray = new int[] {7,4,9,2,5}; // Arrange     
  var minimum = Statistics.GetMinimum(unsortedArray); // Act     
  Assert.AreEqual(2, minimum); // Assert 
}

Guidelines to structuring a unit test

Here are some guidelines you can follow when writing unit tests:

• Keep the number of assertions per unit test to a minimum. A single unit test is testing one thing. Multiple assertions in a single test are fine, but if it’s logical to split the assertions into separate tests, then it’s best to do so.
• Avoid assertion-less tests. These are tests that don’t contain any assertions (or verifications in the case of implementation tests), and are used to test that something works without throwing an exception. I like my unit tests to always check that something worked.
• Don’t repeat assertions that have been covered in existing tests. If a unit test asserts that a result is not null, or that a collection has exactly one item, then subsequent unit tests don’t need to repeat such assertions before asserting additional state.
• Assertions should be situated in the unit tests proper, rather than pulled out into helper methods. If checking the state is a bit complicated and common across many tests, it’s good to write a helper method to check the state. It’s easy to then put the assertions within that helper method, though I find unit tests more readable if they contain the assertions which could check a boolean returned by the helper method.
• The code for arrange, act and assert should be on their own lines, ideally with a new line between each. If you’re asserting that a method returns true, it can be tempting to perform that method call right inside the assert statement. I find unit tests clearer when these are kept separate.

Test Doubles

A unit test is only concerned with testing a single unit, not whether multiple units work correctly together. Some people see a unit as a class, others as a method. Either way, a single unit often requires other objects in order to function correctly. To get the tests to compile, we could build real objects and give them to the unit under test in the same way we would when using the unit in production. By doing this, however, we start leaning away from really just testing a single unit, and couple the test code to more units than it’s concerned with. It can even be impractical to use real objects if they connect to external technologies such as third-party web services, queuing systems or data stores.

This is where test-doubles come in. There are various test-doubles that we can use to stand in for actual objects. The purpose of these is solely to fulfill the requirements of using the unit under test. The following are the test doubles I know of and have used.

Dummies

Dummies are typically values that don’t matter but are needed in order to call a method that we are testing. The value may be irrelevant as it does not affect the test but needs to exist in order to call the method. Out parameters are a common example of this:

[Test] public void GetOccurrences_NewDateTimePattern_HasZeroOccurrences()
{   
  var pattern = new DateTimePattern();
  var dummy;

  var count = pattern.GetOccurrences(out dummy);

  Assert.AreEqual(0, count);
}

Stubs

Stubs are hard coded methods that return an expected answer and don’t care about method arguments or the state of any objects, so don’t function normally. They might be anonymous functions passed directly to a method on the unit we are testing, in which case we are testing that the unit behaves correctly when the function returns that hard coded result. A stub could be a method implemented as a requirement of an interface that the unit under test needs to call. A stub that checks the existence of a file, for example, could always return true if we’re not actually using the file system while testing the unit:

public bool FileExists(string path)
{   
  return true; 
}

Fakes

Fakes are entirely functional objects that usually implement an interface or at least extend an abstract class that the unit under test needs. Fakes are quick and dirty implementations that wouldn’t be used outside of being a unit test double. There are many good examples of fakes, I’ve recently used a fake to implement an IRedisClient interface (data structure store). Rather than actually running Redis, data gets stored in C# data structures in a very simplistic manner. Units being tested that require an IRedisClient to operate can be given an instance of that fake rather than relying on Redis to be running:

public class FakeRedisClient : IRedisClient
{   
  private Dictionary<string, object> _redis = new Dictionary<string,object>();
  // and so on     
  public void AddItemToSet(string setId, string item)
  {
    object obj;
    _redis.TryGetValue(setId, out obj);
    HashSet set = (HashSet)obj;
    if (set == null)
    {
      set = new HashSet();
      _redis[setId] = set;
    }
    set.Add(item);
  }     
  // and so forth 
}

Mocks

Mocks are used to test internal behavior or implementation, rather than state. You use them to verify, for example, certain functions were or were not invoked as a result of calling the method you are testing. Mocking is generally achieved with the help of mocking frameworks such as Moq for.NET. You mock an object from an interface which gives you a concrete object to work with. As part of the setup, you can attach bits of logic or hard coded values in place of methods or properties. This is done so that the mocked object behaves correctly when used by the unit under test. A mocked object can be functionally similar to a fake, and is capable of being used for state testing. I strictly use mocks only for the few occasions I write implementation tests. Fakes are better suited for state testing as they cleanly encapsulate private members such as variables and functions:

[SetUp]
public void SetUp()
{  
  _customer = new Mock();   
  _customer.Setup(c => c.PaymentID).Returns(1); 
}

[Test] 
public void CreateSubscription_NewCustomer_ExistingSubscriptionsAreChecked()
{   
  var service = CreatePaymentSubscriptionService(); 

  var subscription = service.CreateSubscription(_customer);

  _customer.Verify(c => c.GetSubscriptions());
}

So Where do I Stand Personally?

I mostly write tests for the code that I write and fill in missing tests for code that I have access to. Therefore, I’m certainly a white box tester. As for state vs. implementation, I primarily focus on testing state. I like to ensure that a unit is working correctly as observed externally with no assumptions on how it gets work done. This is how other units in a running program will see and interact with it. This sounds a lot like what the result of black box testing would achieve. White box testing gives me the added benefit of making sure all code paths and any obscure edge cases are asserted.

You may find that testing primarily by state leaves the implementation more free to change. Any part of the implementation that is not coupled to the API such as the data structures that are used, the way data is formatted, the keys used to store data and so on, then there will be no mention of these in my state tests. By updating the internal implementation without affecting the API or expected results, you’ll be able to run the unit tests to find the unit still works as it did.

If primarily testing by implementation, the unit tests become tightly coupled to that internal implementation which can make the tests quite brittle. Changing the implementation breaks the tests and requires rewriting them.

That said, a few implementation tests here and there can be useful. For example, having a unit that contains a data structure used as an internal cache for resolving performance concerns. This is purely an implementation decision that doesn’t need to be known by anything outside of the unit. The API of the unit does not expose anything related to the state of this internal cache, so no way to assert it. For peace of mind, you may want to test this cache is being managed well such as clearing out items when appropriate. To do this, it may be better to resort to implementation testing, if you don’t want to expose the existence of the caching in some way to test its state.

I don’t strictly force myself to only test by state and ignore implementation testing. I think it’s good to know what approaches are available and pick the right tool for the job.