Self-Hosted Gateway Design Patterns Discussion

Microsoft Azure provides the default API management platform tool call API Management, which provides rich features to manage and monitor APIs in the Azure environment. However, for the hybrid on-prem/off-prem cloud environment and multi-cloud environment, the organization may want to simplify the API communication to make it more efficient, flexible on executing external workflow, or plugin a security module to manage APIs hosted on-premises or across clouds. 

The self-hosted gateway feature extended from Azure API Management enables those hybrid and multi-cloud API management requirements and allows the organization to customize the API implementation, integration, communication, routing management, monitoring, traffic, and security controls in the cloud environment.  It supports APIs hosted in either on-prem or off-prem across different cloud instances.

In MS Azure, the self-hosted gateway is deployed as a containerized, functionally equivalent, simplified version of the managed gateway in the Azure cloud infrastructure. It is managed from one single APIM instance in Azure. Multiple self-hosted gateways can be aggregated within one APIM instance. The sample self-hosted gateway container can be downloaded from the Microsoft container registry repository. It can be deployed into Azure with Kubernetes infrastructure, or an on-prem environment. 

There are three basic services required by the Azure self-hosted gateway, which are Management plane, Gateway, and Developer portal. For the detail of those feature information, please refer to the link.

In summary, a self-hosted gateway allows API invocation traffic to flow directly to the backend APIs, which optimizes data transfer efficiency, improves communication latency, and enables Azure APIM compliance. It is one simple way to monitor, manage, integrate, composite, and deploy application APIs. 

The aggregated self-hosted gateways aggregated logical structure is as below, High lighted in light blue circles:

The key features comparing self-hosted gateway with regular API gateway managed by APIM are as below:

By looking at those features above, it will be interesting to understand the what core design patterns are used for the self-hosted gateway and how to use those design patterns for the programming purpose. In this article, we will list four typical design patterns used for the self-hosted gateway, and discuss them from the implementation of view. 

Heart Beat Design Pattern

In Azure, self-hosted gateways require outbound TCP/IP connectivity based on 443 HTTPS ports. The self-hosted gateway can be configured via its management plane and send signals back to APIM based on heart-beat time intervals. 

The pattern for the heartbeat will involve two parties, sender and receivers. The sender keeps on sending messages or events to receivers. The receiver will execute the corresponding program or workflows when receiving the sender’s event. If no event is received, the receiver would execute other programs and wait for the next event cycle. 

Usually, the heart-beat pattern is used for connectivity checking purposes. When the connection is lost, the self-hosted gateway could backup the local data, release resources, and wait for the next event cycle. When the connection is restored, the self-hosted gateway will reload the program, restore the process, and update the configuration. The logical process of the heart-beat pattern is as below: 

Pseudocode:

public void heartBeat(HeatBeatPort port)  { 
		while (true) {
			if(heartBeatSwitcher) {
				WatchModuleStatus(iHeartBeatModules ); //System scan on all components
				SendHeartBeatMsg(heartBeatMsg, port); //Send heart-beat message
			} else {
				SuspendHeartBeat(port);  //Send suspend status, if the switch is off
			}
			
			Sleep(timerInterval); //Sleep for the next wakeup
		}
	}


//in Receiver side 
private HeatBeatPort tcpipPORT //443

try ( 
    ServerSocket serverSocket = new ServerSocket(tcpipPORT.getPortNumber());
    Socket clientSocket = serverSocket.accept();
    BufferedReader in = new BufferedReader(
        new InputStreamReader(clientSocket.getInputStream()));
) {
	String inputLine;
	while ((inputLine = in.readLine()) != null) {
		updateHeartbeatModuleStatus(inputLine);
	}
	Sleep(timerInterval); //Sleep for the next wakeup
}

Event Aggregator Pattern 

Self-hosted gateways can be aggregated into one Azure APIM instance. Aggregator in design patterns refers to an entity or a service that collects related data from a set of sub-entity or sub-services. The Aggregator also invokes various sub-processes to collect the required information or execute the required workflow functionality within sub-entities or sub-services.

This pattern provides the benefit when the hosted service needs the data output by combining results from multiple sub-entity or sub-services by one unique transaction or a commend. Each sub-service could be designed in a monolithic architecture and have its own data repository. 

Based on the DRY (Don't repeat yourself) principle, the Aggregator can abstract the logic into composite microservices and aggregate those logics into one service. One typical pattern that implements that purpose is the service façade pattern.

So, for this purpose, the event-based aggregator service can be written and collects data from multiple subservices or microservices, and runs the workflows with business logic, and returns the results. The event aggregate pattern is designed especially by invoking composite microservices with a business workflow while others are preparing the result data simultaneously. Every monolith microservice is designed at a low-end atomic level that can work autonomously and independently. 

Here, we use the event sourcing publish and subscription model to simulate an event aggregate pattern used in a self-hosted gateway service. 

Pseudocode:

//Aggregator Publisher side
public void publishAggregatedEvents(){
	if(eventQueue.isEmpty()){
		System.out.println("No events from publishers to display");
	}else{
		while(!eventQueue.isEmpty()){
			AggregateEvent agEvent = eventQueue.remove();
			String topic = agEvent.getTopic();

			Set<Subscriber> subscribersOfTopic = subscribersTopicMap.get(topic);
			for(Subscriber subscriber : subscribersOfTopic){
				//add broadcasted event to subscribers event queue
				List<AggregateEvent> subscriberEvents = subscriber.getSubscriberEvents();
				subscriberEvents.add(agEvent);
				subscriber.setSubscriberEvents(subscriberEvents);
			}			
		}
	}
}
//Loop in subservices and retrieve all aggregated results based on event JSON payload
public void processAggregatedSubscribers(AggregateEvent agEvent){
	String topic = agEvent.getTopic();
	String payload = agEvent.getPayload();

	Set<Subscriber> subscribersOfTopic = subscribersTopicMap.get(topic);
	for (Subscriber subscriber : subscribersOfTopic) {
		// process subscribes based on event payload JSON object
		List<AggregatedResult> subscriberResults = subscriber.getSubscriberResults(payload);
		processResults(subscriberResults);
	}
}

//Aggregator Subscriber side
@Override
public void handleEvent(final AggregateEvent agEvent) {
    if (EventUtil.isValid(agEvent) && EventUtil.isEnabled(agEvent)) {
  	  String topic = (String) agEvent.getTopic();
  	  String payload = (String) agEvent.getPayload();
        if (StringUtils.isNotEmpty(topic)) {
            if (StringUtils.isNotEmpty(payload)) {
                try {
              	  if(subscribersTopicMap.containsKey(topic)){
              		  onProcessEvent(payload);
              	  }
                } catch (EventFormatException e) {
                    LOG.error(e.getMessage(), e);
                }
            } else {
                LOG.warn("Received an empty event: [{}]", agEvent);
            }
        } else {
            LOG.warn("Received an invalid event: [{}]", agEvent);
        }
    }
}

Proxy Pattern 

Self-hosted gateways can be used as a proxy server for the backend microservices. The Proxy pattern is very similar to Aggregator Design Pattern, but in this case, no aggregation is needed to happen on the client, a different microservice may be invoked based upon the business need. It’s more like a service façade and all client services are behind the interfaces, not exposed to the external clients.

The proxy pattern allows different implementations for the single service interface. It works as a delegator and hides the service implementations from the outside service invokers.  It increases the request communication security but sacrifices the request invocation performance.  

Proxy pattern enhanced the security and access control. Using the delegated representative layer between invokers and clients, it hides the physical service locations and implementations. However, as mentioned above, it increases the time cost and complexity to investigate the runtime issues. Sometimes, it could be a bottleneck in a complex system.

Pseudocode:

public HostedServiceProxy getSimpleProxyClass(String spName) {
    InvocationHandler handler = new MyInvocationHandler(spName);
    Class<?> proxyClass = Proxy.getProxyClass(SimpleProxyService.class.getClassLoader(), SimpleProxyService.class);
    SimpleProxyService sp = (SimpleProxyService) proxyClass.getConstructor(InvocationHandler.class).
                    newInstance(handler);
    return sp;
}
public void proxyTo(SelfHostedRequest req) {
    URI uri = req.getTo().getURI();
    Application application = dataStore.getApplicationForAddress(uri.toString());
    if (application == null) {
        LOG.error("No application for the request:" + req);
        return;
    }
    SelfHostedNode node = loadBalancer.pickSelfHostedNode(application.getHostedNode());
    if (node == null) {
        LOG.error("No available node for the request " + req);
        return;
    }
    try {
        Proxy proxy = getSimpleProxyClass(CONFIGURED_SIMPLE_PROXY_CLASS);
        proxy.setfilter(false);
        proxy.setRecurse(false);
        __beforeHostedServiceProxy(proxy, node);
        proxy.proxyTo(createURI("sip:" + node.getServiceAddress()));
        __afterHostedServiceProxy(proxy, node);
    } catch (HostedProxyException e) {
        LOG.error("Bad proxy address:" + "sip:" + node.getServiceAddress(), e);
    } catch (Exception e) {
        LOG.error("Bad proxy request:" + req, e);
    }
}

Gateway Routing Pattern

Obviously, the self-hosted gateway can be designed as an API gateway which is the design pattern for request routing. The main purpose of the API gateway pattern is request routing. The routing could be a physical address routing or consulting to a naming service for the logical naming routing. The implementation is based on the complexity of the business requirements.  

Besides the routing service, a complex gateway also provides data transformation, protocol bridging, analytics, security controls, and traffic metrics, etc. Let’s don’t be over-architect the solution in this discussion. Self-hosted gateway needs to work as a router for the request routing, a data aggregator to collect data from multiply sub-services, a protocol abstraction layer to high the sub-service API interface, and a centralized error handler to handle the service request process error in general and provide the default behavior for the system exceptions. 

Although the gateway pattern is widely adopted in web application and microservices architecture, it has some drawbacks:

• Increased complexity - the gateway routing map table is hard to be managed and each destination service must be implemented, deployed, and managed. 
• Performance issue due to the additional network communication cost via the routing
• Could be the single point of failure.

Pseudocode:

@Bean
public RouteLocator simpleHostRouteLocator(RouteLocatorBuilder builder, UriConfiguration uriConfiguration) {
  String httpUri = uriConfiguration.getHttpbin();
  return builder.routes()
    .route(p -> p
      .path("/client")
      .filters(f -> f.addRequestHeader("client", "client service"))
      .uri(httpUri))
	.route("user_route", r -> r.path("/su/**")
			.filters(f -> f.stripPrefix(1))
			.uri("lb://SERVICE-USER"))
	.route("content_route", a -> a.path("/ct/**")
			.filters(f -> f.stripPrefix(1))
			.uri("lb://SERVICE-CONTENT"))
	.route("api_route", r -> r.path("/api/**")
			.filters(f -> f.stripPrefix(0))
			.uri("lb://SERVICE-API"))
    .route(p -> p
      .host("*.sfgateway.com")
      .filters(f -> f
        .circuitBreaker(config -> config
          .setName("routingcmd")
          .setFallbackUri("forward:/fallback")))
      .uri(httpUri))
    .build();
}
@Bean
public RouterFunction<ServerResponse> testFunRouterFunction() {
	RouterFunction<ServerResponse> route = RouterFunctions.route(
			RequestPredicates.path("/client"),
			request -> ServerResponse.ok().body(BodyInserters.fromValue("Hello, client service")));
	return route;
}

Conclusion

Here we had a brief introduction to Azure self-hosted gateway, walkthrough gateway key features, and four simple design patterns that its uses. However, in a real gateway project, the design patterns will be much more complex than simple heartbeat, event aggregator, proxy, and gateway routing patterns. This article just wants to highlight the core functions of a gateway and bring up a discussion on the pattern’s benefits and disadvantages. 

In the real world, gateway patterns help microservices runtime communication, however, it could a bottleneck of service invocation in case of a large volume of requests. 

Thanks for reading! Your comments are most welcome.