Comparing XML and JSON Web Services for Output Equality

The other day I had to figure out a way to compare two different web services to see whether a subset of their outputs were equal or not. One of these services (let’s call it the legacy service) is capable of emitting XML as a response, while the other one (a completely new service that is intended to be a replacement for the legacy code) uses JSON. Unfortunately, the output of these two are only similar in a conceptual way: their structures differ a lot, and they use very different ways of structuring data.

I checked out some common frameworks for testing web services, and though all of them were useful for parts of my task, they proved to be hard to use for my complete work. Of course, all these frameworks are too general purpose, while my problem is, I guess, not that of a common use case. At this time I started experimenting with REST Assured – a framework that finally saved me a lot of time and effort. In the below post I will describe how easy it was to get my job done in a relatively short time with this cool framework.

First of all, as I mentioned: the hard part of this task was not the inherent difference between XML and JSON, there are really good tools to overcome this issue. The problem was that the logic by which the two services structured their responses was completely different: same data, different representation. To make things just a little bit worse, the new endpoint is communicating in a secure manner, through https.

I’m not going to share the exact details of the two responses, but the below code represents a simplified (and modified) version of the outputs. First, the legacy XML output:

<countries>
    <country>
           <country_id>10</country_id>
           <providers>
             <provider>
                <id>1</id>
                <name>a_name</name>
             </provider>
           ...
           </providers> 
   </country>
   <country>
           <country_id>11</country_id>
           <providers>
             <provider>
                <id>1</id>
                <name>a_name</name>
             </provider>
           ...
           </providers> 
   </country>
</countries>

And now, the JSON variant:

{ "providers": [
   {
        "id": 1,
        "name": "a_name",
        "active_countries": [10, 11],
        ...
   },
   {
    ...
   }
   ...
]
}

As you can see why the XML document is country-centric, the JSON response takes a different view and puts providers as top level elements.

As JSON is really easy to parse using Jackson, I decided to start with this part and stick to its structure for my model objects (the only data I needed was the provider ID and country IDs for that provider). I was more than happy to see that REST Assured has a way of ignoring https validations, so I didn’t need to create and use self-signed certificates and keystores. This way, it was really easy and convenient to call the service, parse and convert the response, and have the fields I was interested in (less than 50 lines of code):

// ...
public List<Provider> extractNrOfResults() {
    try {
    String responseBody = makeQuery().body().asString();
    Response response = objectMapper().readValue(responseBody, Response.class);
    return responseToProvidersConverter.convert(response);
} catch (Exception e) {
    throw new RequestFailedException(e);
}
}

private RestAssuredResponseImpl makeQuery() {
RestAssuredResponseImpl response = (RestAssuredResponseImpl) 
given()
.relaxedHTTPSValidation()
.headers(createHeaders())
.when()
.body(REQUEST_BODY)
.post(SERVICE_URL);
response.then()
.statusCode(200);

return response;
}

private Map<String, String> createHeaders() {
Map<String, String> headers = new HashMap<>();

headers.put("Content-Type", "application/json");
// ... more headers ...
return headers;
}

Note the relaxedHTTPSValidation() method call in the given() clause.

The first part was ready, but I still needed to extract the data from XML and convert the result to the same format as in the first case. At the beginning, I was thinking about using JAXB for parsing and a custom converter to translate the resulting objects to the format I picked earlier.

Luckily enough, I did not need to do that. REST Assured already has another great feature for efficient data acquisition from XML documents, called XmlPath. One can feed XmlPath with Groovy expressions for efficient data retrieval; this makes it really powerful and easy to understand at the same time. In order to get the data I needed out of the XML document, I only had to use this much code:

private static final String ALL_PROVIDER_IDS_QUERY = "regions.depthFirst().findAll{ it.name() == 'id' }";
private static final String COUNTRY_IDS_BY_PROVIDER_QUERY = "countries.country.findAll { country -> country.providers.provider.any{ it.id == '%d' }}*.country_id";

protected List<Provider> extract(RestAssuredResponseImpl response) {
XmlPath path = new XmlPath(response.asString());

List<Integer> ids = new ArrayList<>(path.getList(XML_ALL_PROVIDER_IDS_QUERY, Integer.class));

return ids.stream()
.map(id -> new Provider(id, getRegionIds(path, id)))
.collect(Collectors.toList());
}

private List<Integer> getRegionIds(XmlPath path, int providerId) {
return path.getList(String.format(COUNTRY_IDS_BY_PROVIDER_QUERY, providerId), Integer.class);
}

As I am not a Groovy guru, I needed to ask for help. Fortunately, I got it pretty fast here. A quick explanation for the fellow Groovy novices: the first query (ALL_PROVIDER_IDS_QUERY) selects all the provider IDs from the whole XML document and returns them in a list. The second one (COUNTRY_IDS_BY_PROVIDER_QUERY) extracts all the country codes for a specific provider. Unbelievable, but even easier than JSON parsing.

The only thing left was to write a good equals method inside Provider and I was done. I really think REST Assured is an awesome tool that can save a lot of time and effort. It’s worth giving it a try next time you need to test web services.