Full Lifecycle API Management Is Dead: Build APIs Following Your Software Development Lifecycle With an Internal Developer Platform

This is an article from DZone's 2023 Software Integration Trend Report.

For more:

Read the Report

As organizations look to enable integration, innovation, and digital experiences through their IT teams, they often build APIs and expose them by leveraging a full-lifecycle API management system. Historically, these API management systems provided tooling such as: 

• Defining an API (e.g., Swagger, OpenAPI Spec, RAML)
• API testing
• API scaffolding and implementation
• Specifications for quota and usage policies/plans
• Documentation
• An API portal

These API management systems were often delivered as a fully integrated stack with a fancy UI, role-based access control, and push-button mechanisms to accomplish the lifecycle management functions. 

While this all sounds very nice, there are some realities we face as organizations look to modernize their application and API delivery engines. An API management platform does not exist in a vacuum. DevOps philosophies have influenced organizational structures, automation, and self-service. Any API management system must fit within a modern development environment that is often multi-language, multi-platform, and multi-cloud. This infrastructure must also fit natively with Git-based deployment workflows (GitOps), including systems built for CI/CD.

Avoid Yet Another Silo (YAS)

Although developer productivity can be difficult to measure, proxy metrics that can be useful include things like the following: 

• Lead time to make code changes in production
• Number of deployments to production per week

Traditionally, developers write code, create services, build APIs, and then hand them off to operations to deploy and operate those services and APIs. The silos between development, infrastructure, security, and network teams often leads to complex synchronization points, handoffs, and a lot of waiting. This slows down code changes and deployments to production. 

Figure 1: Siloed handoffs between teams cause a slowdown in delivery to production

Large monolithic software systems can further this problem by forcing their own silos within each of the organizational silos. They have their own proprietary UIs, require specialized skills or privilege to operate, and are often owned by specific teams. If you need something from the large monolithic software system, you typically need to open a ticket to signal to the team who owns the system that they need to make a change. 

In practice, traditional full lifecycle API management systems create silos by forcing users into an all-or-nothing set of tools for defining, implementing, testing, and exposing APIs even if these differ from what a development team wants to use. These systems are very difficult to automate and integrate with other parts of the software delivery systems, and they are usually guarded by some API management team that is responsible for configuring and deploying APIs. This centralization from both a technology and organizational standpoint creates bottlenecks that slow down delivery in a modern DevOps-minded organization.

Favor Automation Over Point-and-Click UIs

Most traditional full lifecycle API management systems do have some role-centric capabilities, like role-based UIs and tools for specific personas. One principle prevalent in modern DevOps implementations is around eliminating manual or repetitive tasks using automation. We cannot expect users to log into a system that runs tests, a totally different system to manage APIs, and yet another system to do a deployment. 

Figure 2: We should reduce multiple, manual, point-and-click UIs in favor of automation

Ideally, we would automate a lot of these steps so a developer can go to a single self-service UI for anything related to software development and deployment. Any functionality we would like, including traditional API management and each of its "full lifecycle" functionalities, should be automatable. With a lot of the functionality in modern API management locked into proprietary UIs, automation is often very challenging and brittle, if accomplished at all.

The API Lifecycle Is The Software Development Lifecycle

The API lifecycle is often centered around design, implementation, testing, control, and consumption. Does this sound familiar? It should — because it's exactly what we do with any software we write. When developers create APIs, they use software to do so. The API lifecycle is the software development lifecycle. Trying to treat the lifecycle of APIs differently from the rest of our software development practices creates inconsistencies, fragmentation, and friction. 

For example, when we create an API, we may need to develop it, test it, and will probably eventually need to notify users when we need to retire it. We need the same capabilities for internal services, libraries, and other system components. Although there may be some slight differences, should these be separate and different processes? Should these be completely different sets of tools? Trying to duplicate what is already necessary for the software development lifecycle with substandard and proprietary tools specific for API management causes adoption, governance, and bifurcation issues. 

Use an Internal Developer Platform

As organizations attempt to improve developer productivity by shifting left and giving developers more responsibility and control over building and running their services and APIs, we've seen an emergence in platform teams responsible for building workflows and toolchains that enable self-service. These workflows get boiled down to "golden paths" that developers can easily follow and that automate a lot of the tasks around bootstrapping new projects, documenting their software, enforcing access/security policies, and controlling deployment rollouts. 

This developer-focused self-service platform is known as an Internal Developer Platform (IDP) and aims to cover the operational necessities of the entire lifecycle of a service. Although many teams have built their own platforms, there are some good open-source frameworks that go a long way to building an IDP. For example, Backstage is a popular open-source project used to build IDPs.

Platform engineering teams typically have a lot of flexibility picking the best of breed tools for developers that support multiple types of languages and developer frameworks. Plus, these tools can be composed through automation and don't rely on proprietary vendor UIs. Platform engineering teams also typically build their platform around container technology that can be used across multiple clusters and stretch into on-premises deployments as well as the public cloud. These IDPs insulate from vendor lock-in whether that's a particular public cloud or vendor. 

For example, here's a very common scenario that I've run into numerous times: An organization bought into a full-lifecycle API management vendor and finds itself in a situation where their modernization efforts are centered around containers and Kubernetes, GitOps, and CI/CD. They find the API management vendor may have strong tools around API design; however, runtime execution, the API portal, and analytics features are lagging, outdated, or cannot be automated with the rest of the container platform via GitOps. They often wish to use a different API gateway technology based on more modern open-source proxies like Envoy Proxy but are locked into a tightly integrated yet outdated gateway technology with their current vendor. 

Instead, these organizations should opt to use newer proxy technologies, select more developer-friendly API testing tools, tie API analytics into their existing streaming and analytics efforts, and rely on tools like Backstage to tie all of this together. Doing so, they would reduce silos centered around vendor products, leverage best-of-breed tools, and automate these tools in a way that preserves governance and prescribed guard rails. These platforms can then support complex deployment strategies like multi-cluster, hybrid, and multi-cloud deployments. 

Conclusion 

Managing APIs will continue to be an important aspect of software development, but it doesn't happen in a vacuum. Large monolithic full lifecycle API management stacks are outdated, don't fit in with modern development practices, and cause silos when we are trying to break down silos. Choosing the best-of-bread tools for API development and policy management allows us to build a powerful software development platform (an IDP) that improves developer productivity, reduces lock-in, and allows organizations to deploy APIs and services across containers and cloud infrastructure whether on-premises or any public cloud.

This is an article from DZone's 2023 Software Integration Trend Report.

For more:

Read the Report