Giving Your Legacy Applications an API Facelift

Let’s face it: In today’s modern world of cloud and containers, there are still thousands of legacy applications that were not written with an API-first approach. Some legacy systems can still provide tremendous value today, but the means for accessing them are completely out of date, thus rendering them almost useless. Kong, an open-source API gateway, can help with this!

In this post, we’ll walk through creating an extremely simple—but powerful—Kong Gateway Lua plugin to showcase how you could put an API in front of a legacy application that you could otherwise only access through a CLI.

To draw a parallel with a real-world scenario, this is the case of one of my customers (who shall remain nameless). The company has a need to connect to a disparate array of legacy endpoints and expose them via REST APIs to build more modern applications. 

Accessing such an endpoint system via a CLI is just one of the many applications my customer has to connect to. To avoid a full-blown integration implementation, a reusable plugin pattern such as the one we are talking about in this post might just do the trick.

Let’s dig in.

What’s in a Name?

I have affectionately named this Kong Gateway plugin Kronos. No specific reason other than I like the name, it works well as alliteration, and it reminds me of the movie The Incredibles—if you didn’t see the movie, Kronos is the name of the plot for a giant robot falling from the sky and laying waste to a city, but there is no need to be dramatic. Or maybe there is.

Enter Pig Latin

In this example, we will imagine our legacy application is a Pig Latin translator. The application is written in Perl, which traditionally works like this:

[kong-master:~]# perl piglatin.pl "Giving Your Legacy Applications an API Facelift with Kong"
ivingGay ourYay egacyLay Applicationsway anway APIway aceliftFay ithway ongKay
[kong-master:~]#

I never learned Pig Latin, so this is pretty cool. The problem is, to run this application, we have to be in front of (or SSH into) the system so we can type out the command. Not cool anymore. 

My customer’s legacy applications are definitely much more complex than our Pig Latin translator, and that is perfectly fine. However, how we expose the Perl program in this tutorial is a blueprint anyone can use for exposing legacy applications that lack a REST API.

What You’ll Need

Requirements:

• docker-compose
• If you’d like to try out the Pig Latin translator, install the Perl PigLatin module using cpan Lingua::PigLatin  (you’ll do this in the Kong shell, courtesy of Gojira)
• Gojira
• Access to the source code
• A Linux or Mac environment

If you haven’t written Kong Gateway plugins before, a great environment for developing and testing them is Gojira. The instructions for setting up Gojira are outside the scope of this post, but rest assured that the setup is pretty straightforward. As you can see in the image below, I am running Gojira locally on my Mac.

david.lamotta@Davids-MacBook-Pro-13 gojira % gojira roar
 
                 _,-}}-._
                /\   }  /\
               _|(O\_ _/O)
             _|/  (__''__)
           _|\/    WVVVVW    Let Me Fight!
          \ _\     \MMMM/_
        _|\_\     _ '---; \_
   /\   \ _\/      \_   /   \
  / (    _\/     \   \  |'VVV
 (  '-,._\_.(      'VVV /
  \         /   _) /   _)
   '....--''\__vvv)\__vvv)      ldb
 
david.lamotta@Davids-MacBook-Pro-13 gojira %

Once Gojira is up and running, we’ll need to install a couple of Kronos dependencies. You can do that in the Kong shell, which you can run by executing gojira shell.

david.lamotta@Davids-MacBook-Pro-13 gojira % gojira shell
[kong-master:/kong]#

In this shell, you’ll need to install lua-resty-exec, luasocket, and sockexec. You can easily install both Lua modules via luarocks install lua-resty-exec and luarocks install luasocket. Verify the installation with a show command like this.

[kong-master:/kong]# luarocks show lua-resty-exec                          

lua-resty-exec 3.0.3-0 - Run external programs in OpenResty without spawning a shell

License:      	MIT
Homepage:     	https://github.com/jprjr/lua-resty-exec
Installed in: 	/build/luarocks

Modules:
	resty.exec (/build/luarocks/share/lua/5.1/resty/exec.lua)
	resty.exec.socket (/build/luarocks/share/lua/5.1/resty/exec/socket.lua)

Depends on:
	lua >= 5.1 (using 5.1-1)
	netstring >= 1.0.6 (using 1.0.6-0)

Indirectly pulling:
	netstring (using 1.0.6-0)

[kong-master:/kong]#

[kong-master:/kong]# luarocks show luasocket     
 
LuaSocket 3.0rc1-2 - Network support for the Lua language
 
LuaSocket is a Lua extension library that is composed by two parts: a C core
that provides support for the TCP and UDP transport layers, and a set of Lua
modules that add support for functionality commonly needed by applications that
deal with the Internet.
 
License:       MIT
Homepage:     http://luaforge.net/projects/luasocket/
Installed in: /build/luarocks
 
Modules:
 ltn12 (/build/luarocks/share/lua/5.1/ltn12.lua)
 mime (/build/luarocks/share/lua/5.1/mime.lua)
 mime.core (/build/luarocks/lib/lua/5.1/mime/core.so)
 socket (/build/luarocks/share/lua/5.1/socket.lua)
 socket.core (/build/luarocks/lib/lua/5.1/socket/core.so)
 socket.ftp (/build/luarocks/share/lua/5.1/socket/ftp.lua)
 socket.headers (/build/luarocks/share/lua/5.1/socket/headers.lua)
 socket.http (/build/luarocks/share/lua/5.1/socket/http.lua)
 socket.serial (/build/luarocks/lib/lua/5.1/socket/serial.so)
 socket.smtp (/build/luarocks/share/lua/5.1/socket/smtp.lua)
 socket.tp (/build/luarocks/share/lua/5.1/socket/tp.lua)
 socket.unix (/build/luarocks/lib/lua/5.1/socket/unix.so)
 socket.url (/build/luarocks/share/lua/5.1/socket/url.lua)
 
Depends on:
 lua >= 5.1 (using 5.1-1)
 
[kong-master:/kong]#

The last step to run processes through Kronos on your host is to have a Unix domain socket on your file system. For that, you’ll have to download the version of sockexec that matches your architecture. 

Note: If you ever get to run this in a production-type environment, please make sure you have checks and balances so you don’t end up doing more harm than good. 

In this example post, Gojira is running an instance of Kong Gateway in a containerized environment, so I am leveraging the ARM-based build.

Extract the sockexec tarball on your local machine, and copy sockexec and sockexec.client to the /bin directory in your Kong shell. The result should look like this (don’t forget to chmod +x both files).

[kong-master:/kong]# ls -la /bin | grep socket*
-rwxr-xr-x 1 root root   58776 Sep  7 14:16 sockexec
-rwxr-xr-x 1 root root   33936 Sep  7 14:16 sockexec.client
[kong-master:/kong]#

To easily copy the files from your local development environment (i.e., your laptop) to the running Kong container, you can run a quick python one-liner from the directory where we extracted the tarball. Once the http server is running in that directory, bring the files over with something like curl, directly from the Kong shell. There are many ways to climb a tree, and this is just one of them.

The Plugin

That preamble to getting Kronos ready to go is by far the most time-consuming piece. As you’ll see below, the actual implementation of Kronos is rather simple. Granted, there is plenty of room for improvement, but that can be an exercise for you, the reader. For now, the whole intent of this post is to show you the art of the possible and the powerful nature of Kong’s plugin architecture. 

There are some great tutorials out there for writing Kong Gateway plugins, and the official Kong documentation is wonderful to get the details on plugin development, too. 

As mentioned at the beginning of this post, Kronos is a barebones, simple plugin. At its core, it is basically acting as a passthrough proxy to the host, where you can execute commands directly on it, courtesy of our prerequisites stated earlier. Kronos uses a strategy pattern, so you can specify the implementation that is touching the legacy endpoint. We will see this in more detail when we explore the source code.

At the very least, Kong plugins need two Lua files: handler.lua and schema.lua. The first file contains the logic for your plugin, and the second contains configuration parameters you can send to the plugin. In our case, there is also a third file, piglatin.lua, which implements the “piglatin” strategy.

First off is schema.lua, a simple schema with only one configurable parameter: strategy. It is via this config that we can specify what we will execute on our legacy endpoint. Additionally, we specify that we can’t apply Kronos to service, only to a consumer or a route.

-- Copyright (c) 2021 Kong, Inc.
-- All rights reserved.
--
-- Use of this source code is governed by a BSD-style
-- license that can be found in the LICENSE file or at
-- https://opensource.org/licenses/BSD-3-Clause
--
-- Author: David La Motta
--
local typedefs = require "kong.db.schema.typedefs"
-- Grab pluginname from module name
local plugin_name = ({...})[1]:match("^kong%.plugins%.([^%.]+)")
 
local schema = {
 name = plugin_name,
 fields = {   
   -- this plugin will only be applied to Consumers or Routes
   { consumer = typedefs.no_service },     
   -- this plugin will only run within Nginx HTTP module
   { protocols = typedefs.protocols_http },
   { config = {
       type = "record",
       fields = {
         { strategy = { type = "string", required = true } },
       },
     },
   },
 }
}
 
return schema

handler.lua has the delegation implementation, which is a fancy way of saying that our handler is delegating calls to the appropriate strategy. In this example, we only have a single strategy called piglatin. Should the caller send anything other than that, we are going to get an error. For the purists out there, you should definitely implement defensive programming and make sure that the strategy is not nil before moving forward. You’ve been forewarned.

-- Copyright (c) 2021 Kong, Inc.
-- All rights reserved.
--
-- Use of this source code is governed by a BSD-style
-- license that can be found in the LICENSE file or at
-- https://opensource.org/licenses/BSD-3-Clause
--
-- Author: David La Motta
--
local kronos = {
 VERSION  = "1.0.0",
 PRIORITY = 10,
}
 
local strategies = {
 piglatin = require "kong.plugins.kronos.piglatin"
}
 
function kronos:init_worker()
 kong.log("kronos:init_worker")
end
 
function kronos:access(config)
 local strategy = strategies[config.strategy]
 local args = kong.request.get_header("X-Args")
 local code, res = strategy.exec(args)
 
 if code == 500 then
   kong.log(err)
 end 
 
 return kong.response.exit(code, res)
end
 
return kronos

We implemented the one strategy we have been talking about in this post in piglatin.lua below. The most interesting parts are the declaration of the variable prog, and the body of the method _M.exec(args). prog depends on that Unix domain socket we mentioned earlier, which we will use to launch an external process from Kronos. In our implementation, that external process is Perl, pointing to our piglatin.pl file using the arguments passed in by the caller.

-- Copyright (c) 2021 Kong, Inc.
-- All rights reserved.
--
-- Use of this source code is governed by a BSD-style
-- license that can be found in the LICENSE file or at
-- https://opensource.org/licenses/BSD-3-Clause
--
-- Author: David La Motta
--
local exec = require'resty.exec'
local prog = exec.new('/var/run/exec.sock')
local _M = {}
 
function _M.exec(args)
 local cmd = { "perl", "/root/piglatin.pl", args }
 prog.argv = cmd
 local res, err = prog()
 
 if err then
   return 500, err
 end 
 
 return 200, res.stdout
end
 
return _M

The path for the socket is entirely up to you, but it must exist (and it must be readable by the lua-resty-exec process), or else Kronos will not work. To create that socket, you’ll have to execute the following:

[kong-master:/kong]# sockexec /var/run/exec.sock &
[kong-master:/kong]# chmod 777 /var/run/exec.sock

Last, but not least, is the piglatin.pl file, which is the application we are exposing via a REST API. As you can see in piglatin.lua, our Perl file is in the /root directory. You are welcome to place it wherever you’d like; just make sure you modify the path in your strategy file accordingly. piglatin.pl is extremely simple, as you can see—this is a mock legacy application, after all.

# Copyright (c) 2021 Kong, Inc.
# All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file or at
# https://opensource.org/licenses/BSD-3-Clause
#
# Author: David La Motta
#
use Lingua::PigLatin 'piglatin';
print piglatin(@ARGV[0]);

And voila!

Suppose you were writing this to access your own legacy endpoints. In that case, you could have multiple strategies, more configuration parameters OR no strategies, and no configuration parameters at all! The art of the possible is just that: This is your canvas, and you are free to do with this as your requirements demand, without losing sight of providing the most secure means for doing so.

Now that we have the code behind us, let’s call out one last step so you can tweak your code, reload Kong, rinse and repeat. One of the great advantages of Gojira is that you can edit your Lua files locally, and upon making any changes, all you have to do is reload Kong. The only magic you have to do to get this working is to mount the volume as you bring up Gojira (you’ll do this once and then operate in the shell and your code, most of the time).

david.lamotta@Davids-MacBook-Pro-13 gojira % gojira up --volume /Users/david.lamotta/kong/plugins/kronos:/kong/kong/plugins/kronos

From there, you are off to the races. After modifying your code, simply reload Kong. You’ll have to tell Kong to add the Kronos plugin explicitly, so don’t ignore the export statement.

[kong-master:/kong]# export KONG_PLUGINS=bundled,kronos
[kong-master:/kong]# kong reload
Kong reloaded
[kong-master:/kong]# kong health
nginx.......running
 
Kong is healthy at /kong/servroot
[kong-master:/kong]#

Setting Things Up

Wonderful! We have our prerequisites set and our code is written. Now it is time to set things up. You might be wondering where the service, route, and plugin are. However, as you noticed in our handler.lua file, we effectively intercept the request, delegating the strategy module to the execution of a command on the host and returning the output. This means there is no service! What we have to do is create a route and apply Kronos to that route.

david.lamotta@Davids-MacBook-Pro-13 kronos % curl -i -X POST http://localhost:50777/routes --data "name=kronos" --data "hosts[]=kronos.dev"
HTTP/1.1 201 Created
Date: Fri, 10 Sep 2021 14:27:16 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Content-Length: 442
X-Kong-Admin-Latency: 13
Server: kong/2.5.0
 
{"snis":null,"https_redirect_status_code":426,"id":"8bb677ae-2e13-4a2f-af05-5dc4fa6bf846","tags":null,"path_handling":"v0","created_at":1631284036,"hosts":["kronos.dev"],"destinations":null,"paths":null,"response_buffering":true,"sources":null,"service":null,"methods":null,"headers":null,"preserve_host":false,"strip_path":true,"regex_priority":0,"updated_at":1631284036,"protocols":["http","https"],"request_buffering":true,"name":"kronos"}                                                                                             david.lamotta@Davids-MacBook-Pro-13 kronos %

Let’s apply the plugin to that route:

david.lamotta@Davids-MacBook-Pro-13 kronos % curl -i -X POST http://localhost:50777/routes/kronos/plugins --data "name=kronos" --data "config.strategy=piglatin"
HTTP/1.1 201 Created
Date: Fri, 10 Sep 2021 14:28:15 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: *
Content-Length: 274
X-Kong-Admin-Latency: 11
Server: kong/2.5.0
 
{"consumer":null,"tags":null,"service":null,"config":{"strategy":"piglatin"},"created_at":1631284095,"protocols":["grpc","grpcs","http","https"],"name":"kronos","enabled":true,"id":"ce04bb02-0904-4d3d-a83b-c01b550201e9","route":{"id":"8bb677ae-2e13-4a2f-af05-5dc4fa6bf846"}}     
david.lamotta@Davids-MacBook-Pro-13 kronos %

We are leveraging the Kong Admin API to create the route and apply the plugin to that route. If you’d like to know what port the Admin API is running on your local machine, you can find out through Gojira.

david.lamotta@Davids-MacBook-Pro-13 gojira % gojira port 8001
0.0.0.0:50777
david.lamotta@Davids-MacBook-Pro-13 gojira % gojira port 8000
0.0.0.0:50776
david.lamotta@Davids-MacBook-Pro-13 gojira %

Taking Kronos for a Test Drive

Kong offers a terrific tool called Insomnia for designing, testing, and debugging APIs. We will be using Insomnia to test out Kronos and finally get to use our Pig Latin translator! Let’s first check the route and the plugin using Insomnia:

Our route is ready. Now, let’s check the plugin. Below, we configured our Kronos plugin on the route with the same ID as the route shown above.

Last but not least, the fun part. Let’s get some Pig Latin translation through an API call, shall we? 

Notice X-Args in the header? We are sending the text we want to translate in the request’s header—again, not to any service. In our code in handler.lua, we are grabbing the contents of that parameter and passing it right down to the strategy implementer (i.e., piglatin.lua). We take the standard output from the execution and pass it right back up the stack to the caller.

In a production deployment of a plugin such as Kronos, we would be remiss if we didn’t state that you should put extra care into the flexibility placed on the parameters passed in and the commands executed on the host. That is to say, be mindful of the protection you should place on your commands and parameters so that we stop anything malicious in its tracks.

In Closing

To quote the Spider-Man movie, “With great power comes great responsibility.” I bet you can start imagining how you can effectively use Kronos to do all sorts of things now that you have access to the underlying host. Without any checks and balances, a rogue strategy could lay waste to an underlying host. This is where the strategy pattern and Kong’s plugin ecosystem come into play. 

You can externalize implementing a strategy to an application owner, so they carry the burden of responsibility. 

I cannot stress how useful something like Kronos can be for exposing legacy systems that might only have a CLI or any other antiquated interface—I hope you would agree. If anything, this post shows you the power of Kong and the flexibility you have at your fingertips for exposing the world via APIs.

Future Work

Accessing Windows hosts is another ball of wax, but it is possible to make it work with this framework. Stay tuned for a future post where we explore doing things on a Windows machine through Kronos. A Windows strategy has a nice ring to it.

Another improvement that we could apply to Kronos is that of executing a strategy asynchronously. This improvement could reduce latency upon making the API request. For this, we would have to expand the schema and potentially introduce callbacks. This topic might be an exciting third installment to the series.