User-Friendly API Publishing and Testing With Retrofit

Any web service needs to export their public API if consumers want to make the best use of that service. A developer-friendly approach to do so if you work in the Java ecosystem is to package DTOs and endpoint interfaces in an API jar file and use the Retrofit framework to create type-safe clients for integration testing. This article discusses a complete example.

If you’ve worked in enterprise Java projects you will remember good old Web Services Description Language, the XML based format for describing network services by IBM and Microsoft. Maybe you still work with it? WSDL and its twin XML Schema are among those W3C standards that seasoned developers love to hate. Its specification files are not very human readable, let alone human writable. Fortunately you don't have to. They can be generated by your server endpoint and fed straight into a code generator to create transfer objects (DTOs) and service stubs.

The purpose of a contract specification document is to make the outward-facing parts of your service known for the benefit of teams that use your service. No complex application can do without it, especially microservice environments that are maintained by remote teams. If the verbosity and complexity of WSDL is the bathwater, then an unambiguous service specification is the baby. We cannot afford to throw the latter out. Industry standards that must remain implementation agnostic and broadly applicable have fallen out of fashion because of their unavoidable complexity. But we must have an alternative.

If you develop both clients and servers in the same ecosystem, you have the luxury to be more opinionated about the protocols and toolsets. This can lead to a simpler and more developer-friendly way to publish your API. In this article I will show you an example REST service that exposes a single endpoint and a decoupled test project that queries that service. The approach is two-fold:

• Publish your API by packaging a jar file containing your data transfer objects as POJOs and your API endpoints as Java interfaces. 
• Both the REST server and the test project depend on this API jar. Consumers of the service use the interfaces to build client proxies with the Retrofit framework. The REST server only references the DTOs.

The Project

You can find all source code on my GitLab page:

git clone git@gitlab.com:jsprengers/spring-retrofit-demo.git

The maven project consists of a parent project with service, api and integration sub-projects.

• api contains data transfer objects and REST controller specifications.
• service is the Springboot REST service. It depends on the api project for the DTOs.
• integration contains only integration tests. It depends on api for the DTO and the REST endpoint specifications.

The Service Project

The endpoint returns and consumes a Person DTO defined in the API project. A simple memory-based data access object mimics persistent storage between backend calls. A Basic Authentication implementation distinguishes two roles (read and write) and the users 'user' and 'admin'. admin can execute PUT, POST and DELETE methods while user can only do GET requests. Passwords are provided as environment variables on startup, with "nosecret" and "secret" as default values for the user and admin logins, respectively. It's a test project, after all.

@RestController
@RequestMapping("api/person")
@RequiredArgsConstructor
@Slf4j
public class PersonController {

    @Autowired
    private final PersonDAO personDAO;

    @GetMapping
    List<Person> getAll(@RequestParam(value = "fields", required = false) String fields) {
        return personDAO.getAll(fields);
    }

    @GetMapping("/{id}")
    Person getPersonById(@PathVariable("id") String id, @RequestParam(value = "fields", required = false) String fields) {
        return personDAO.getById(id, fields).orElseThrow(() -> {
            throw new NotFoundException("No such ID: " + id);
        });
    }

    @PostMapping
    void createPerson(@RequestBody Person person) {
        if (personDAO.getById(person.getId(), null).isPresent()) {
            throw new IllegalArgumentException("Person with ID already exists: " + person.getId());
        }
        log.info("Storing person with id {}", person.getId());
        personDAO.put(person);
    }

    @PutMapping
    void upsertPerson(@RequestBody Person person) {
        personDAO.put(person);
    }

    @DeleteMapping("/{id}")
    void deletePerson(@PathVariable("id") String id) {
        personDAO.deleteById(id);
    }

}

The API Project

The API project consists of the data transfer object in your business domain, i.e. the Person POJO. The Lombok framework minimizes boilerplate. They can contain documentation and (de)serialization hints in the form of JSON annotations if you so wish.

@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder(toBuilder = true)
public class Person {
    private String id;
    private String name;
    private String address;
    private String email;
}

The second part of your service contract are the controller endpoints, expressed as Java interfaces. These are basically the Controller methods stripped of their bodies. This is where the Retrofit framework comes in. It provides a set of annotations to decorate these interfaces that we will turn into network proxies to the services they target. These annotations are highly analogous to what you do on the server-side controllers.

public interface PersonAPIClient {

    @GET("api/person")
    Call<List<Person>> getAll(@Query("fields") String fields);

    @GET("api/person/{id}")
    Call<Person> getPersonById(@Path("id") String id, @Query("fields") String fields);

    @POST("api/person")
    Call<Void> createPerson(@Body Person person);

    @PUT("api/person")
    Call<Void> upsertPerson(@Body Person person);

    @DELETE("api/person/{id}")
    Call<Void> deletePerson(@Path("id") String id);
}

Notice the return types. As you will see later, these interfaces are the templates for implementations that return a parameterized Call object, which is a proxy to the underlying network client that reads the response body and gives information about the status of the Http request. Note that because of this uniform Call return type, you cannot let your REST controllers implement the API interface. We could have interesting discussions on whether it's a good thing to have this tight coupling, but it's a moot point. As it stands, you have to maintain the interfaces manually and separately.

The Integration Project

This project demonstrates a test that starts and queries a REST service, with only a source dependency on the service’s published API. The package stage of the build pushes a Docker image of the runnable service to your local repository using the jib-maven-plugin The integration test setup pulls that Docker image and runs a container using the Testcontainers framework.

public class PersonAPIContainerizedIntegrationTest {

    private static AppContainer container;
    private static PersonAPIClient userClient;
    private static PersonAPIClient adminClient;

    @BeforeAll
    public static void initialize() {
        container = new AppContainer();
        container.startAndWait();
        // The port for the localhost endpoint is available 
        // through container.getFirstMappedPort()
        RetrofitClientFactory retrofitClientFactory = 
           new RetrofitClientFactory(container.getFirstMappedPort());
        userClient = retrofitClientFactory.authenticatedClient("user","nosecret");
        adminClient = retrofitClientFactory.authenticatedClient("admin", "secret");
    }

    @AfterAll public static void shutdown() {
        if (container != null && container.isRunning())
            container.stop();
    }
   [ ... ]
}

AppContainer is an implementation of a GenericContainer in the TestContainer framework. It starts the containerized REST server that was built and pushed to the local repository during the build of the service project.

public class AppContainer extends GenericContainer<AppContainer> {

    public AppContainer() {
        //The dockerized springboot app run on port 8080, the only port that is exposed by the image
        super(DockerImageName.parse("spring-retrofit-test-server:LATEST"));
        withExposedPorts(8080);
    }

    protected void startAndWait(){
        this.start();
        //The container port 8080 is mapped to a free port, available through getFirstMappedPort()
        // it blocks until the api/person is available.
        this.waitingFor(new HttpWaitStrategy()
           .forPath("api/person/")
           .forPort(getFirstMappedPort()));
    }
} 

Creating a REST Client With Retrofit

The Retrofit instance is the factory that creates REST clients from interfaces. You need to configure it using a builder pattern which takes at the very least the base URL of your service. We use Google GSON as a JSON converter. We also need to configure the underlying OkHttpClient library for basic authentication in order to test access restrictions of the user and admin roles, so we can initialise different clients for each role. The port of the URL is only known after the container has started, so is has to be configurable.

With Retrofit.create you make a proxy for your service endpoint (PersonApiClient in the api project). It’s convenient to have a single interface per corresponding controller class, but that is not required.

public class RetrofitClientFactory {
    private final int port;

    PersonAPIClient authenticatedClient(String username, String password) {
        OkHttpClient okHttpClient = new OkHttpClient.Builder().authenticator(
                (route, response) -> response.request().newBuilder().header("Authorization", Credentials.basic(username, password))
                        .build()).build();
        Retrofit retrofit = new Retrofit.Builder()
                .baseUrl(String.format("http://localhost:%d/", port)).
                        addConverterFactory(GsonConverterFactory.create())
                .client(okHttpClient)
                .build();
        return retrofit.create(PersonAPIClient.class);
    }
} 

Using the Retrofit Client

@Test
    public void EntityLifeCycleHappyFlow() throws IOException {
       
        executeCall(adminClient.createPerson(Person.builder().id("42").name("Jane").build()));
        executeCall(adminClient.createPerson(Person.builder().id("43").name("Jack").build()));
        assertThat(executeCall(userClient.getPersonById("42", null)).body().getName()).isEqualTo("Jane");
        assertThat(executeCall(userClient.getPersonById("43", null)).body().getName()).isEqualTo("Jack");

        Response<List<Person>> response = executeCall(userClient.getAll(null));
        assertThat(response.body()).hasSize(2);

        executeCall(adminClient.upsertPerson(Person.builder().id("42").name("Jane").address("London").build()));
        Person jane = executeCall(userClient.getPersonById("42", "address,dateofBirth")).body();
        assertThat(jane.getAddress()).isEqualTo("London");
        executeCall(adminClient.deletePerson("42"));
        assertThat(userClient.getAll(null).execute().body()).hasSize(1);
    }

  private <T> Response<T> executeCall(Call<T> call) throws IOException {
        Response<T> response = call.execute();
        if (!response.isSuccessful()) {
            fail("response returned " + response.errorBody().string());
        }
        return response;
    }

Here you can see a Retrofit client in action. Programming against an interface makes for clear, concise and type-safe coding. Remember each method returns a Call object that you must execute() to give you a Response. The executeCall is a convenience method to prevent unhelpful RuntimeExceptions when a call is not successful.

Speaking of which, testing unhappy paths is clean and easy. The Response object gives you all the info you need.

// the user role is not allowed to do POST requests 
assertThat(userClient.createPerson(Person.builder().id("42").name("Jane").build()).execute().code()).isEqualTo(403);

// there is already a user with id 42.
Response<Void> personExists = adminClient.createPerson(Person.builder().id("42").name("Jane").build()).execute();
assertThat(personExists.code()).isEqualTo(400);
assertThat(personExists.errorBody().string()).isEqualTo("Person with ID already exists: 42");

//user name cannot be blank
Response<Void> incompletePost = adminClient.createPerson(Person.builder().id("44").name(null).build()).execute();
assertThat(incompletePost.code()).isEqualTo(400);
assertThat(incompletePost.errorBody().string()).isEqualTo("Person name cannot be null");

I hope you have enjoyed this article and found it useful. There is more to learn about the Retrofit utility in the full documentation that I didn't cover here. Calls can also be queued asynchronously, passing it a callback handler rather than the simpler thread-blocking strategy we used here. Retrofit is by no means limited to integration tests. You can also use it in production code that accesses other services.

All in all I find it very intuitive and a real joy to work with when writing integration tests, much more user-friendly than a contender like RestAssured.