REST Without JSON: The Future of IoT Protocols

With the rise of web-based APIs, we have come to think of REST (Representational State Transfer) as being synonymous with JSON over HTTP. Unsurprisingly, JSON has supplanted XML as the data format of choice for the web. While early IoT technologies have embraced the JSON/HTTP mix, that could soon be changing. The concept of REST will survive, but JSON and HTTP may not be the common language of IoT data interchange for much longer.

You may also like: How to Choose the Right IoT Connectivity Protocol for Your Connected Device 

At its core, REST is an architectural pattern for uniformly accessing and modifying a resource. One entity (the server) is the authority over the current state of an object. Other entities may request a “representation” of the current object, and may also send requests to create, modify, or delete the object. The current popular REST model uses URIs to identify objects (“/lamp/1234”), HTTP verbs to specify an action, and JSON to represent the object. To fetch an object, a client may send an HTTP request to “GET /lamp/1234”. The server may respond with an HTTP 200 and a body containing JSON data.

The HTTP/JSON model is deeply entrenched in web APIs, and its popularity has naturally seeped into IoT technologies. Samsung, Nest, and Apple have all published APIs that rely on JSON over HTTP, but this early trend will wane. While the REST model works well for the distributed network that makes up the new IoT world, HTTP 1.1 and JSON are not the right fit.

What’s Wrong With JSON?

When JavaScript legend Douglas Crockford introduced the JSON format, he was interested in specifying a format that eased data interactions between web applications and JavaScript-based clients. Because it’s a lightweight alternative to XML, JSON quickly gained traction among web developers, and later reached a more general audience.

Several features of JSON make it a great candidate for general purpose data interchange. First, it is schemaless; as long as the JSON is well-formed, it is valid. Second, JSON supports a minimal and straightforward set of data types: strings, numbers, booleans, objects, arrays, and a null value. Third, the data is represented in JavaScript syntax, which makes it both human-readable and easily parseable. One would be hard-pressed to find a popular programming language that does not have at least one JSON parser.

These features make JSON a useful general-purpose format, but the typical use cases for IoT may cause us to doubt JSON’s appropriateness for the embedded systems that together make up the smart device landscape. IoT devices typically need to optimize along the following lines:

• Keep network traffic small and fast.
• Minimize the amount of raw computation for network encoding and decoding.
• Use only small amounts of memory and storage.

Devices may run with less than a megabyte of memory or storage, and often run on small batteries. For power consumption reasons, they may only be on the Wi-Fi network for a few seconds at a time, and sometimes only a few times a day. Even high-end hub devices are unlikely to have more than 25MB of storage at their disposal. For these devices, efficiency is key, especially when it comes to networking.

JSON is not the best candidate for meeting these requirements. First, in spite of its claims to leanness, JSON is not a space-efficient encoding. All data are expressed as ASCII strings, often with copious white space added. Every label field must be repeated in its entirety for each occurrence. Binary data must be escaped, though there is no standard method for doing so in JSON.

This leads to a second issue with JSON. The simplicity of the data format introduces complexity in implementation. JSON’s simple types rarely match the types typically used in IoT programming. While languages like C support a broad array of numeric types, JSON’s only numeric type is number. The official JSON specification, ECMA-404, does not even define the maximum size of a number field. This means that a JSON consumer must do a fair amount of examination to determine which underlying type best matches a given number. This is complicated by the fact that two or more fields with the same apparent structure and field names may contain different “types” of numbers. The field “age” may in one occurrence be an unsigned positive integer and in another be a floating point.

The problem is compounded by JSON’s lack of schema. Arrays can contain any number of types, and there are no constraints on how the fields of an object are used or whether they are used consistently. Developers rely only on convention to determine what data a JSON structure will contain. Finally, there is the problem of interpreting a JSON data structure. Fields are essentially unordered (except for arrays). Valid JSON may, as noted above, contain arbitrary data that violates expectations, and it is up to the parser to solve any given data structure. Strategies used for efficient field-level processing generally don’t work well with JSON. Practically speaking, that means parsing an entire object and storing the results in memory.

JSON is clearly not the best technology for data encoding. HTTP 1.1, the other half of the ubiquitous REST implementation, does not come out looking much better.

What’s Wrong With HTTP?

HTTP 1.1 has served web developers well. It is flexible, straightforward, enjoys broad implementation, and has a huge developer base. But the HTTP faults that have irked web developers for years may have an even bigger impact on IoT developers.

Like JSON, HTTP tends toward the bloated side. HTTP headers are a good example. As plain text strings with no compression of any sort, they bloat the network protocol.

Network usage is another one of HTTP’s shortfalls. The original HTTP specification was designed around the idea of short-lived network connections. The client opens a connection and then requests a page, the server delivers it, and the connection is closed. But the average web page now may fetch over a dozen resources at once. HTTP 1.1 introduced some capabilities for keeping connections opened and reusable for short periods of time, but HTTP essentially remained focused on short-lived connections.

Consider the networking aspect of an IoT device. Establishing a connection is expensive in terms of power and time, especially with SSL/TLS negotiation included; every added connection brings along a substantial computational hit. Repeatedly opening heavyweight network connections is an unnecessary drain on resources.

In the IoT world, every byte sent and received from an embedded device takes its toll on performance. A good IoT protocol not only makes it easy for the developer to send the right information, but it also reduces the burden on the device and its network. The HTTP payload model is great for IoT, but a better protocol would streamline security, optimize transmission sizes, and focus on multiplexing requests and responses over long-lived network connections.

The Future Is Binary

REST is a good model for IoT. Each device can easily make its state information available and can standardize on a way to create, read, update, and delete that data. Developers can quickly build a mental REST model for many IoT devices. Get the state of the lightbulb: it is off. Send a request to turn it on. Get the current temperature from the space heater: it is too hot. Send a lower target temperature. The model seems to intuitively match the problem space.

But what is to be done about JSON and HTTP? IoT developers need REST without needless bloat.

For JSON, the future of IoT is bleak: a host of better-suited encodings are flooding the space. Apache Thrift and Google’s Protocol Buffers (Protobuf) each provide binary encodings better suited for constrained devices, and both have the advantage of automatically enforced schemas. CoAP, an emerging standard for IoT communication, defines an encoding called CBOR. CBOR is self-describing, and the encoding is focused on producing small message sizes. Even the venerable ASN.1 family of encodings may be getting a new IoT spin. All of these provide encoding characteristics better suited to embedded devices than JSON.

For HTTP, the story may play out differently. True, it will face some competition; for example, CoAP defines a concise REST-like transport protocol that is a compelling alternative to HTTP 1.1. But growing out of Google’s SPDY efforts, the HTTP/2 standard indicates that HTTP may have solved its own problems.

HTTP/2 shows a renewed interest in network performance. Headers in HTTP/2 are efficiently encoded. The protocol supports multiplexing many streams of data over one connection, as well as server-initiated pushes, and the reconstruction of the protocol maintains SSL/TLS as a central piece. One SSL/TLS negotiation can then protect multiple streams of data, thus reducing the setup overhead, but maintaining a high degree of security.

In addition to HTTP/2 and CoAP, the emerging QUIC protocol may also gain traction among resource-constrained devices. QUIC, also a Google protocol drawing from SPDY, trades TCP for UDP. By removing some of TCP’s connection management overhead, QUIC aims to reduce latency, particularly during initial establishment of a network connection.

Because QUIC and HTTP/2 are based on a similar protocol stack, the competition between the two is not a zero-sum game. Both are well-designed and are likely to gain acceptance in the emerging IoT space.

The Turning Tide

The REST model is a strong fit for IoT. However, the traditional REST implementation of JSON over HTTP is ill-fitting at best. JSON’s string-oriented payloads are no match for binary encodings when it comes to data transmission in terms of speed and ease of parsing. Encodings like CBOR and Protobuf are compelling alternatives to JSON.

In contrast, the HTTP/2 specification indicates that HTTP may remain the application protocol of choice. And its emerging sister protocol, QUIC, will complement and strengthen the position of web protocols in the IoT space.

This article first appeared in DZone's Guide to the Internet of Things – 2015 Edition. 

Further Reading

How to Choose the Right IoT Connectivity Protocol for Your Connected Device 

Protocols and Standard Behind IoT: CoAP, BLE, MQTT, DDS