Testing HTTP APIs With Supertest

You know tests are good for you. You probably even write unit tests and measure the code coverage of your business logic. Congrats! However, at that layer, you're only writing tests for yourself, your teammates, or others consuming your code-level interfaces.

But what about your web-level interface? Your HTTP API is the interface you give others who depend on your code without even being able to see it (like the good ol’, pre-open source days… right?).

That outbound API needs at least as much love and attention (if not more!) than the internal, code-level APIs you write for yourself and friends.

Surface Cleaning Your API

HTTP APIs are the web-level “surface area” of your application. External developers — including the browser-client team — depend on this surface to build their code on top of it. Making sure it’s dependable and does what it promises is invaluable to giving others a solid foundation to build upon.

Recently, I’ve been working on testing the Web Annotation Protocol which was designed as part of the Web Annotation Working Group at the W3C. Unlike the code you’re writing now, this protocol started its life as a specification and, as it has grown and changed, has been implemented to make sure the spec can do what it says it can.

To test the Web Annotation Protocol, I started by implementing it within the Web Platform Tests system frequently used to test browser-side JavaScript APIs specified by various W3C Working Groups. I sat down to write some Python, receive some HTTP requests, and return some JSON.

However, things quickly got bothersome as testing the internal Python API was only part of the puzzle. I needed to be sure curl and the browsers would get the correct output from my API, and I needed the ability to test it from “across the Web” where the various intermediaries (proxies, coffee shops, etc.) might interfere.

So I ended up testing my test server via a thing called SuperTest.

Meet SuperTest

SuperTest is built on the fabulous Node and browser HTTP client called SuperAgent.

Here’s what SuperAgent code to send a POST request looks like (cribbing from their opening example):

request
   .post('/api/pet')
   .send({ name: 'Manny', species: 'cat' })
   .set('X-API-Key', 'foobar')
   .set('Accept', 'application/json')
   .end(function(err, res){
     if (err || !res.ok) {
       alert('Oh no! error');
     } else {
       alert('yay got ' + JSON.stringify(res.body));
     }
   });

SuperTest extends that basic (and wonderfully obvious) API with a singular .expect() function. With that singular addition, you can expect all kinds of things!

But First, Mix In Some Chai(.js)

Tasty tests (for me, anyhow) start with some Mocha and some Chai!

• Mocha is a test framework for Node and web browsers.
• Chai is a Behavior and/or Test Driven Development assertion library. Mix them together with SuperTest, and you have a test-driven HTTP API development process.

Here’s one of the tests (along with the basic setup) that I’ve cribbed from my Web Annotation Protocol Tester code:

// pull in assert.isTrue() and the rest
var assert = require('chai').assert;
var request = require('supertest');
var uuid = require('node-uuid');

var host_url = 'http://localhost:8080'
var container_url = host_url + '/annotations/';

describe('MUSTs', function() {
  describe('4. Annotation Containers', function() {
    it.skip('An Annotation Server MUST provide one or more Containers');
    it('MUST end in a "/" character', function(done) {
      assert.isTrue(container_url[container_url.length-1] === '/');
      done();
    });
  });

  describe('4.1 Container Retrieval', function() {
    container = request(container_url);
    it('MUST support GET, HEAD, and OPTIONs methods', function(done) {
      container
        .get('')
        .expect('Allow', /GET/)
        .expect('Allow', /HEAD/)
        .expect('Allow', /OPTIONS/)
        .expect(200, done);
    });
  });
});

When run, this outputs a command line report that looks similar to this screenshot:

One thing I like about the mix of Mocha, Chai, and SuperTest is this “spec” report output. In this case, I’ve mapped the various test suites directly to the sections of the Web Annotation Protocol and written SuperTest requests to generate HTTP client requests in order to test the server that claims to implement the protocol.

The first of these example tests uses some simple Chai assert.isTrue() goodness to check the shape of a URL. The second uses SuperTest to check that the Allow header is set and that it includes GET, HEAD, and OPTIONS methods. This means the server is claiming it allows those methods on the requested resource. There are other tests which test each of those actual methods.

I built the Web Annotation Protocol Tester to test my personal implementations of the Web Annotation Protocol. The code and runtime environment is deliberately separate from the the server and intentionally designed to test any implementation of the Web Annotation Protocol. I didn’t want to put these tests near the server code for fear of “cheating” or mixing them with code-level tests.

The different sort of separation of concerns works as a handy forcing agent to induce good habits on the protocol implementer. It’s TDD for APIs.

Setting Expectations

You can quaff more Mocha and Chai by digging into their documentation. Today, we’ll focus on SuperTest’s superness!

The SuperTest API adds the single .expect() method to the SuperAgent API. This single method, though, has a wide range of uses, which we’ll explore in the next few sections.

In all these cases, the fn function receives the current err. If no function is passed, an error will be thrown. However, if there is an .end() function, errors will be passed to that and can be re-thrown or passed to Chai’s done() method.

.expect(status[, fn])

Asserting the response status code is something you’ll do quite often. Use the number of the HTTP Status Code for the value of status.

Check out httpstat.us or httpstatus.es or http.cat if you need help deciding which status to check (or if you just need a great cat photo to display along with the error message).

For instance:

.expect(status, body[, fn])

As in the previous section, status is a number, and body can be either a string (for an exact match scenario) or a regular expression (which is likely more useful).

This one can be very handy for testing error code messages, such as:

request(base_url).get(broken_url).expect(404, /not found/ig);

Or you may just want to be sure a certain key is returned in the JSON object if the response was successful: 

request(base_url).get(broken_url).expect(201, /created/ig);

You can also do an exact match on the parsed JSON object (assuming you’re using JSON). See the next section for an example.

.expect(body[, fn])

In this case, you can focus your error message on the problems with the content of the response. For example:

request(base_url).get('users').expect(/total/, function() {
  throw new Error('Users list must have a `total` key);
});

Often, though, you’ll want to do more than a singe RegEx check on the response content. Hang in there, that option’s around the corner!

.expect(field, value[, fn])

Asserting a header field and its value is one of the most common API tests. Here you can use a string equivalency test or a RegEx. Which one you pick will really just depend on the header you’re checking. We saw the RegEx option in use with the Allow headers in the Web Annotation Protocol example. Here’s a scenario where you might want an exact match:

request(base_url).get('users').expect('Content-Type', 'application/json', function() {
  throw new Error('Default response MUST be in `application/json`);
});

.expect(function(res) {})

This is the catch-all .expect option for checking the request according to anything not handled by the earlier examples. You get the whole request object sent down from SuperAgent. Which means, your test function will have access to the raw response text (res.text), the parsed response content (res.body), the headers (res.header as an array with headers lowercased), the media type (res.type), the character set (res.charset), and of course the status.

This is the most obvious place to do multiple key existence checking in your JSON documents:

request(base_url).get('users').expect(function(res) {
  if (!('total' in res.body) && !('first' in res.body)) {
    throw new Error('Missing pagination information and hypermedia controls');
  }
});

Here’s How it .end(fn)s

Now that all those expectations are set in your tests, you can make some clean console output in conclusion. If you add an .end() method to your chain of .expect() calls, it will get whatever error is thrown handed down. In addition to the error, this method also gets the response for one last processing, logging, or whatever.

Additionally, it serves as an obvious statement that your code will perform the request and expectations you’ve set and then invoke fn(err, res).

request(base_url).get('users')
  .expect('Content-Type', 'application/json', function() {
    throw new Error('Default response MUST be in `application/json`);
  })
  .expect(200)
  .expect(function(res) {
    if (!('total' in res.body) && !('first' in res.body)) {
      throw new Error('Missing pagination information and hypermedia controls');
    }
  })
  .expect(function(err, res) {
    // in this case, just rethrow it via Chai
    if (err) return done(err);
    // or if successful, just be done
    done();
  });

SuperTest Your API!

So, the next time you’re building an API and you want to be sure it’s meaningful or doing that hypermedia thing, grab a can of SuperTest. It’s proven to find errors you didn’t know your API had and in at least half the time it takes to run those tests over and over using curl or a browser and reading the output with your eyes.