Hybrid Integration — Ground-to-Cloud Patterns We're Seeing in the Wild

After years of talking about ‘digital transformation,’ the relevant CIOs and analysts like Gartner are no longer talking about overnight change or quickly replacing legacy systems.

But that’s not to say change is glacial. Thousands of applications power the modern enterprise and many are cloud applications and IoT devices that communicate with each other and with on-prem systems via APIs (application programming interfaces). Today, APIs are novel only to Geoffrey Moore’s ‘laggards.’

What we find interesting, however, is the shift to positioning HIP as a product. Not an integration strategy or a conceptual framework, but a thing you can buy.

The challenge: most of the enterprise customers we work with already have some of the pieces of a HIP in place. If a HIP was just one more box to install atop a server rack, the conversation might be different. But as far as we see, from big banks to fintechs like American Express to pharma to iconic consumer brands like Bacardi rarely want to buy an entire HIP all at once. 

What’s more, we often get asked, “Where does Cloud Elements fit in a HIP?” In these cases, we have to first ask exactly what this particular SI or this particular architect means by ‘a HIP’ before we begin to answer. That said, more and more we see two particular patterns emerge among our largest enterprise users. Before we get to patterns though, a simplified look at the pieces of a HIP:

There are nearly two dozen components in the Gartner HIP diagram; when we talk about it with customers, we try to simplify and start with just three pieces:

1. On-Premises (on-prem) ESB or MOM middleware: this includes ETL database integrations, orchestrated integrations (data + processes) among multiple standalone applications, and EDI, which we include in ‘on-prem’ because EDI platforms are often owned and managed on-prem.

2. Gateway: the technology that allows the enterprise to write, publish, manage and/or monetize APIs for “ground-to-cloud” orchestrated integrations and 3rd party access.

3. Cloud Services: platform services from major cloud providers (such as AWS, Azure, and Google) fall into this category, along with orchestrated integrations between two cloud applications and devices/IoT.

When the challenge is ground-to-ground integration or EDI file transfer, we only need to think about one or two of these key components. The trickier challenge that we see includes all three: ground-to-cloud integrations. 

For enterprise IT orgs and their SI partners trying to, say, connect on-prem ERPs with cloud apps—whether to provide better inventory data for sellers or to better manage eCommerce transaction data and fulfillment—here are the two patterns we’re seeing most frequently:

Orchestrate From the ESB

As long as the ESB has a secure method to call out from the on-prem environment where it lives, the integration developer can simply call the specific connector (Element Instance in our world) that’s connected to the cloud service instance. The developer can write any mappings/ transformations as well as process orchestration logic in the ESB they’re already using.

• But why write to the connector (Element) vs writing directly to the API? Perhaps the endpoint is SOAP/XML and the developer doesn’t want to unpack the XML file. Or perhaps the developer doesn’t want to learn a new set of unique error codes and API methods. Or perhaps the developer doesn’t want to rewrite token management for this particular API’s nuanced requirements. For our customers, Elements normalize endpoints to consistent REST/JSON calls which make life easier and reduce development time & maintenance.

Create a Reusable API Connector for an API You Own

More common for integrations involving custom applications, this pattern explicitly involves the creation/ publication of an API for the target on-prem/ owned cloud application plus the development of a private Element to connect to that API (via Element Builder). Across the enterprise, the Element is now available for the first developer as well as any after her to use, along with any other integration content in Cloud Elements, like Virtual Data Resources (VDRs) or Formula-as-a-Resource (process-based APIs).

• If you’re publishing an API or otherwise using an APIM solution, why use an Element? Aside from the fact that the API itself is well-managed and potentially public (which means you could monetize it in the future), using an Element quickly augments existing APIs, adding consumer-friendly features that many APIs today lack. Perhaps you’ve already published an API but it does not offer eventing/webhooks or bulk operations; instead of rebuilding the API, which may handle other traffic, the Element requires no changes to the API itself and uses out-of-the-box eventing and bulk frameworks as well as standardized calls and discovery APIs to add the needed functionality.

Want to go deeper on hybrid integration scenarios? You can check out our solution brief and Point-of-View for all things hybrid integration, especially ground-to-cloud scenarios.