Why API Schemas Are Better Than Connectors for Integration

Why Enterprise Application Integration Platforms Have Traditionally Used Connectors

A connector (often called an SDK) is a piece of code that wraps an external API so that it can communicate with your application. Huh? Ok, well, let’s run through a practical example.

Say you wanted to update a contact in your CRM system every time there was a change to the same contact in your ERP (or accounting system). The providers of both the CRM and ERP system document their APIs on their websites. This documentation describes all of the operations available in the API, the method by which you should connect to the API, security, and other settings.

However, this is just documentation. It doesn’t actually provide any code to implement the API in your integration. A developer typically requires a connector that wraps all of the operations of the API and exposes them as methods that they can call in their code. In our example, the developer would require a connector for each of the APIs that will be consumed and the connector will expose the operations as methods such as getContact and updateContact.

Most of the Enterprise Application Integration (EAI) platform software industries use connectors to facilitate communication between the software vendors application and an API. The connector is specific to the EAI platform in question and wraps the API in a language that the integration platform understands, making all of the operations of the API accessible to the user of the application.

Why We Killed the Connector

What’s wrong with connectors? They go out of date. Often. Really often. You see, publishers frequently change their APIs, oftentimes with little to no warning. With a connector, you are reliant on the integration platform vendor to:

• have a connector for each API you would like to integrate;
• proactively monitor when publishers make changes to their API; and
• update the connector to support changes to the API.

What’s worse is that connectors are usually coded by hand. That makes creating and updating them time-consuming and error-prone. Maintaining connectors for even a few hundred APIs is a nightmare, yet there are tens of thousands of public APIs, and they are growing at an exponential rate. Creating and maintaining a connector for each of them is near impossible.

The Connector Is Dead. Long Live the API Schema.

With the introduction of TORO Integrate v3, we eliminated the dependency on connectors. Instead, you can integrate directly with any API by simply importing an API schema that describes the API in a machine and human-readable format. TORO Integrate parses the API schema and automatically generates a service for each operation in the API that can then be invoked immediately or incorporated into a more extensive integration service. All without writing a single line of code.

If the API changes, just import the publisher’s latest schema, and you’re ready to go again. There is no need to wait in the hope that the vendor of your application integration platform ships a new connector to incorporate the latest API changes.

What Is an API Schema?

An API schema is a resource generated from an API (application programming interface) DL (description language). A DL is a structured description of a RESTful web API that can be read by a human and processed by a machine. Usually, they are defined in JSON but can also be in YAML, XML or even Markdown. So when we refer to an API schema, we are referring to the JSON (YAML or XML) resource of an API DL.

API schemas provide machine-readable structured documentation that, together with regular documentation and an API explorer environment (or development sandbox), make an API more discoverable. API schemas have several other benefits for the publisher of an API including streamlining the development process and making an API more maintainable. We will go into more details of the benefits of an API first development methodology using API schemas in another article.

Okay, so there is just one format for API schemas, right?

Nope. However, the industry is getting there. As you can imagine there wasn’t a consensus on the definition of API DLs from day one. In fact, for quite a while there were API definition wars from competing formats. These have primarily been won now by a specification called OpenAPI. Most of the industry heavyweights, including those that used to promote their own competing formats, have now joined the OpenAPI bandwagon.

Given that OpenAPI has primarily emerged as the industry standard API DL, we won’t go into the advantages and disadvantages of all the competing formats. However, here are some of the major formats you may see promoted:

OpenAPI

This is an industry alliance that has formed around an open source API DL project called Swagger. Swagger was donated by Smartbear Software to the   OpenAPI Initiative and is governed under the Linux Foundation. You can still   see many APIs using or referring to the Swagger definition. This typically   refers to a Swagger v2.x API definition. From v3.0, the Swagger definition
                                                 became known as OpenAPI.

RAML

RAML was developed by Mulesoft and was a modeling language designed to be suitable to document both RESTful web APIs as well as SOAP and RPC
                                                 protocols. Mulesoft is now a member of the OpenAPI Initiative.

API Blueprint

API Blueprint was developed by Apiary (now part of Oracle) and is a
                                                documentation-focused API DL defined in Markdown. Oracle + Apiary is now
                                                a member of the OpenAPI Initiative.

WADL

WADL (Web Application Description Language) uses XML and focuses on  machine readability. WADL attempted to be the REST equivalent of SOAP;
                                                however, WADL never really took off and is not widely adopted.

Fortunately, TORO Integrate supports the major API DL formats as well as APIs defined in the older SOAP protocol. This means that no matter what API DL format is used, you should be able to import it to Integrate, automatically generate the services for all the operations of the API, and start integrating.

Cool, I’m on Board, but what if the API I want to consume doesn’t publish an API schema?

Companies and services including Slack, Bitbucket, Xero, Marketo, BigCommerce, eBay, Box, Yammer, Lyft, SendGrid, Zuora, Insightly, and Avalara already make API schemas publicly available. API publishers want to make their APIs more discoverable, and as API schemas facilitate this, it is within their interest to publish an API schema.

However, what do you do if the API you want to consume doesn’t publish an API schema?

One option is to read the publisher's API documentation and manually implement the operations you want to consume yourself. This is easy enough to do with TORO Integrate as making an ad hoc connection to a RESTful web API can be made by navigating through a few steps of a wizard. However, this is not convenient if there are dozens of operations you would like to use.

TORO Docs to OpenAPI Creates Schemas for APIs Without Schemas

Fortunately, API documentation is typically generated from templates, so the formatting is consistent. We built a Chrome extension called TORO Docs to OpenAPI that leverages this consistent formatting of API documentation to map the CSS classes contained within the page to the operations of an API. The extension then crawls the page and generates an OpenAPI compliant schema. TORO Docs to OpenAPI is free to download and use, doesn’t require any other software, and the schemas generated by it can be used for any purpose. You can download TORO Docs to OpenAPI yourself and give it a try.

Finally, there are over 1,200 API schemas, and growing every day, in the TORO Marketplace. Those created with TORO Docs to OpenAPI can be opened and edited directly within the extension. All of the schemas in the TORO Marketplace can be downloaded to TORO Integrate, including the free Desktop Edition.

Get Started Consuming an API Using API Schemas

To consume an API using an API schema, follow these steps:

Get TORO Integrate

Search the TORO Marketplace under the “API schemas” category to see if a schema already exists for the API you would like to consume.

Download the schema and automatically generate the services for all of the operations in the API.

You can now invoke the service directly or drag and drop it into a more extensive integration service.

If the schema doesn’t exist in the TORO Marketplace and the API publisher doesn’t make one publicly available, you can create your own using TORO Docs to OpenAPI.