Testing Golang With httptest

Go, often referred to as Golang, is a popular programming language built by Google. Its design and structure help you write efficient, reliable, and high-performing programs. Often used for web servers and rest APIs, Go offers the same performance as other low-level languages like C++ while also making sure the language itself is easy to understand with a good development experience.

Go’s httptest package is a useful resource for automating your server testing process to ensure that your web server or REST API works as expected. Automating server testing not only helps you test whether your code works as expected; it also reduces the time spent on testing and is especially useful for regression testing. The httptest package is also useful for testing HTTP clients that make outbound requests to remote servers.

The httptest package was primarily built to test your Go HTTP handlers using net/http, with which it works smoothly. It can also be extended. httptest can serve as a drop-in replacement for your third-party integrations as a stub, and it can be easily adapted to your local development environment during testing.

This article provides an overview of how to use httptest to test your Go handlers in a web server or REST API and to test HTTP clients. 

What Is httptest?

As mentioned earlier, httptest, or net/http/httptest in full, is a standard library for writing constructive and unit tests for your handlers. It provides ways to mock requests to your HTTP handlers and eliminates the need of having to run the server. On the other hand, if you have an HTTP client that makes requests to a remote server, you can mock the server responses using something like SpeedScale and test your client.

httptest - How It Works

Before looking at the test package, it’s important to first understand how the HTTP handler package itself works and how your requests are processed.

HTTP Handler Package

The standard library http package net/http has a client and a server interface. The server is basically a group of handlers. When you send a request to an endpoint or a server path, the handler intercepts this request, and based on the request, it returns a specific response. 

Below is a simple interface of a handler (http.Handler):

type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

The ServeHTTP takes in ResponseWriter and Request. The Request object holds the incoming request from the client, and the ResponseWriter can be used to create a response.

Here is an example of a simple handler:

// With no ServeHTTP

func handler(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello, World!"))
}

// With ServeHTTP

type home struct {}

func (h *home) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello, World!"))
}

The first method is more common than the second. It’s less confusing and clearer to declare your handler as a function.

When the handler takes a request, you can compose a response after carrying out whatever needs to be done using the `ResponseWriter` interface. This process can either be reading from the database, third-party services, static server data, or processing a file. In the above code, the line w.Write([]byte("Hello, World!")) sends the response.

Testing HTTP Handlers with httptest

Even though the handlers are just functions, you can’t write unit tests for them the usual way. The hindrance comes from the parameters of the handler function with types ResponseWriter and Request. These are constructed automatically by the http library when a request is received by the server. 

So how do you construct a ResponseWriter and Request object to test the handler? This is where httptest comes in.

httptest has two methods: NewRequest and NewRecorder, which help simplify the process of running tests against your handlers. NewRequest mocks a request that would be used to serve your handler. NewRecorder is a drop-in replacement for ResponseWriter and is used to process and compare the response with the expected output.

Here’s a simple example of the httptest methods, an instruction to print the response status code:

import (
    "fmt"
    "net/http"
    "net/http/httptest"
)

func handler(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello, World\n"))
}

func main() {
    req := httptest.NewRequest("GET", "http://google.com", nil)
    w := httptest.NewRecorder()
    handler(w, req)
    resp := w.Result()
    fmt.Println(resp.StatusCode)
}

The line below allows you to create a new mock request for your server. You are passing it to your handler as your request object:

req := httptest.NewRequest("GET", "http://google.com", nil)

The next line is your ResponseWriter interface and records all the responses from the handler:

w := httptest.NewRecorder()

You can now use these variables to call the handler function:

handler(w, req)

Once the request has been fulfilled, these lines let you see the results and the response details of the request: 

resp := w.Result()
fmt.Println(resp.StatusCode)

Let’s now build a complete example. First, create a new Go project and create a file named server.go:

package main

import (
    "fmt"
    "log"
    "net/http"
    "net/url"
)


func RequestHandler(w http.ResponseWriter, r *http.Request) {
    query, err := url.ParseQuery(r.URL.RawQuery)
  
    if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        fmt.Fprintf(w, "Bad request")
        return
    }

    name := query.Get("name")

    if len(name) == 0 {
        w.WriteHeader(http.StatusBadRequest)
        fmt.Fprintf(w, "You must supply a name")
        return

    }

    w.WriteHeader(http.StatusOK)
    fmt.Fprintf(w, "Hello %s", name)
}

func main() {
    http.HandleFunc("/greet", RequestHandler)
    log.Fatal(http.ListenAndServe(":3030", nil))
}

The above code creates a handler that returns a greetings message. The name query parameter is used to create a greeting, which is returned as a response. Run the code and send a request to http://localhost:3030/greet with a name parameter, for example, http://localhost:3030/greet?name=john to see the response.

To test this server, create a file server_test.go and add the following code:

package main

import (
    "io/ioutil"
    "net/http"
    "net/http/httptest"
    "testing"
)



func TestRequestHandler(t *testing.T) {
    expected := "Hello john"
    req := httptest.NewRequest(http.MethodGet, "/greet?name=john", nil)
    w := httptest.NewRecorder()
    RequestHandler(w, req)
    res := w.Result()
    defer res.Body.Close()

    data, err := ioutil.ReadAll(res.Body)

    if err != nil {
        t.Errorf("Error: %v", err)
    }

    if string(data) != expected {
        t.Errorf("Expected Hello john but got %v", string(data))
    }

}

The expected variable holds the expected response of the server. The NewRequest method creates a mock request to /greet with a name parameter. The handler then responds with the appropriate data, which is then validated against the expected value. Run the test with go test, and you should see it pass.

Testing HTTP Clients With httptest

Another important use case of httptest is to test HTTP clients. Whereas HTTP servers intake requests and churn out a response, HTTP clients sit on the other end, making requests to a server and accepting responses from it. Testing the clients is trickier since they depend on an external server. Imagine a scenario where your client makes a request to a third-party service, and you wish to test your client against all types of responses returned by the third-party service; however, it’s not in your control to decide how the third-party service will respond. This is where the NewServer function of httptest comes into play.

The NewServer method creates a mock server that returns the response you want. You can use it to mimic the response of a third-party system. Let’s see an example in action.

Create a new Go project and install the required dependencies:

mkdir httptest-client && cd httptest-client
go init example/user/httptest
go get github.com/pkg/errors

Create a file called client.go. Here, you’ll define the client:

package main

import (
    "io/ioutil"
    "net/http"
    "fmt"
    "github.com/pkg/errors"
)

type Client struct {
    url string
}

func NewClient(url string) Client {
    return Client{url}
}

func (c Client) MakeRequest() (string, error) {
    res, err := http.Get(c.url + "/users")

    if err != nil {
        return "", errors.Wrap(err, "An error occured while making the request")
    }

    defer res.Body.Close()

    out, err := ioutil.ReadAll(res.Body)

    if err != nil {
        return "", errors.Wrap(err, "An error occured when reading the response")
    }

    return string(out), nil
}



func main() {
    client := NewClient("https://gorest.co.in/public/v2/")
    resp, err := client.MakeRequest()

    if err != nil {
        panic(err)
    }

    fmt.Println(resp)
}

The client simply makes an API call to https://gorest.co.in/public/v2/users, which returns some data. The response is then printed to the console.

To test the client, create a file called client_test.go:

package main

import (
    "fmt"
    "net/http"
    "net/http/httptest"
    "strings"
    "testing"
)

func TestClientUpperCase(t *testing.T) {
    expected := "{'data': 'dummy'}"
  
    svr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, expected)
    }))

    defer svr.Close()

    c := NewClient(svr.URL)
    res, err := c.MakeRequest()

    if err != nil {
        t.Errorf("expected err to be nil got %v", err)
    }

    res = strings.TrimSpace(res)

    if res != expected {
        t.Errorf("expected res to be %s got %s", expected, res)
    }
}

Here, the expected variable holds the expected result that the client must return. In this case, the client returns whatever it gets from the server intact, but you might have some processing done before returning the data.

The next line creates the mock server with a handler that returns the expected result:

svr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, expected)
}))

You can change the response value here to test the client against different responses. Remember to change the expected value as well.

Finally, a client is constructed against this mock server, and the request is made:

c := NewClient(svr.URL)
res, err := c.MakeRequest()

Run the test with go test to see the results.

Conclusion

Unit tests are crucial for validating the proper working of any application. Testing HTTP servers or clients is tricky because of the dependence on external services. To test a server, you need a client and vice versa. The httptest package solves this problem by mocking requests and responses. The lightweight and fast nature of httptest makes it an easy way to test your code. This article showed you how to use httptest to test your API handlers and HTTP clients.