Combining gRPC With Guice

gRPC

gRPC is a high-performance protocol for Remote Procedure Calls over HTTP/2. It is mainly used for communication between micro-services, but it can also be used for requests from end-users using browsers or mobile devices just like REST or GraphQL. gRPC was designed by Google, and open-source implementation libraries are available for several platforms and programming languages, including Java.

Quite a unique feature of gRPC is streaming requests and responses: when defining a gRPC procedure, we can indicate that instead of just 1 request message, the client will send a stream of request messages. Similarly, we can indicate that the server will respond with a stream of response messages:

service MyService {
	rpc unary(Request) returns (Response) {}
	rpc streamingClient(stream Request) returns (Response) {}
	rpc streamingServer(Request) returns (stream Response) {}
	rpc biDiStreaming(stream Request) returns (stream Response) {}
}

Request and response streams are completely independent of each other: response messages don't need to be correlated with specific request messages, and a server does not need to wait for his client's stream to finish to start the response stream.

Guice

Guice is a lightweight dependency injections framework for Java also developed by Google. It follows the old Unix principle "do one thing well": it's nothing more than dependency injection and, as such, can be used in multiple environments: servlet apps, custom server apps (including, for example gRPC servers), standalone desktop apps, etc.

One of the most important features of dependency injection frameworks is scoping: when our code requires an object to be injected, the framework may reuse instances associated with the given context. Most of the readers are probably familiar with the concept of servlet scopes: @RequestScoped and @SessionScoped in Guice, @RequestScope, and @SessionScope in Spring. For example, when an injection of an EntityManager or a DB transaction needs to happen, this usually must be an instance associated with the currently handled HttpServletRequest. (side note: in Guice, servlet scopes are not a part of the core framework: they are provided as an extension as they would not make sense in non-servlet apps).

In this article, I will describe what scopes are provided by grpc-scopes lib and explain when and how to use them.

So What Exactly Is a Scope?

Generally speaking, a Scope is an object that knows where to look for and where to store objects associated with some given context. For example, before requesting a new JDBC Connection from a DataSource, @RequestScope may first check if maybe there already is a connection stored in some attribute of the HttpServletRequest being currently handled: if yes, then just inject this stored one. Only otherwise, ask for a new one from the DataSource, then store it under the given attribute for future injections and finally inject it as requested. More formally, in Guice, a Scope is defined as follows:

public interface Scope {
	public <T> Provider<T> scope(Key<T> key, Provider<T> unscoped);

	// javadocs and other boilerplate methods omitted
}

So, for example a simplified implementation of the request scope's scope(...) method could look like this:

public <T> Provider<T> scope(Key<T> key, Provider<T> unscoped) {
	return () -> {
		HttpServletRequest request = getCurrentRequest();
		T instance = (T) request.getAttribute(key.toString());
		if (instance == null) {
			instance = unscoped.get();
			request.setAttribute(key.toString(), instance);
		}
		return instance;
	};
}

(getCurrentRequest() may work, for example, in conjunction with some Filter that stores newly incoming requests on some static ThreadLocal var. Note however that the above implementation has several issues that are not resolved here for simplicity of the scoping concept demonstration)

What Scopes May Be Useful in gRPC Services?

RPC servers expose several procedures and each procedure may be called by several clients. Furthermore, each client may issue several RPC calls concurrently (either to multiple or a single procedure). Naturally, it makes sense for a server to scope injections in the context of a single RPC call. In grpc-scopes lib, this Scope is simply called rpcScope⧉ .

In the case of most stateless RPC systems, rpcScope would be sufficient on its own. However gRPC streaming complicates things quite a bit: streaming calls may last very long: and it is not uncommon for stable micro-services to have to stream RPC calls lasting for several hours. Furthermore, there may be several minute-long pauses between subsequent messages in streams. Altogether this means that rpcScope is not suitable for scoping injections of objects that are intended to be short-lived or not retained when not in active use. For example, transactions should usually last well below a second, while retaining pooled objects such as JDBC Connections may dramatically degrade server performance. A natural solution to this situation is to introduce another scope that would span over the processing of a single message from a request stream.

Java gRPC implementation deals with streams in an asynchronous style: user code gets a callback each time a new message arrives, so the new scope could span over each such single callback invocation. However, a message arrival is not the only callback that a user service code may receive during a lifetime of an RPC call: when dealing with a stream from a peer, both server and client code need to provide an implementation of the StreamObserver⧉  interface to receive stream event callbacks:

public interface StreamObserver<V> {
	void onNext(V value);      // next message arrived
	void onError(Throwable t); // error occurred (on server side this may only be cancellation)
	void onCompleted();        // the other side indicated end of their stream

	// javadocs omitted, method comments added for the purpose of this article
}

On the server side we may also optionally register via a ServerCallStreamObserver⧉  to receive additional callbacks:

public abstract class ServerCallStreamObserver<RespT> extends CallStreamObserver<RespT> {
	public abstract void setOnCancelHandler(Runnable onCancelHandler);
	public abstract void setOnReadyHandler(Runnable onReadyHandler);
	public void setOnCloseHandler(Runnable onCloseHandler) {...}

	// javadocs and other methods omitted
}

"onCancel(...)" is roughly speaking a duplication of onError(...), and "onReady(...)" is called to indicate that the other side is ready to receive more messages (in case of bi-di procedures) after getting temporarily clogged, finally "onClose(...)" is called after the server successfully flushes all response messages in the given call and closes the underlying HTTP/2 stream.

Servers may need to react in different ways to each such event: they may for example need to commit a transaction in "onClose()" and roll back it in "onCancel(...)" etc. To be able to perform such actions, the corresponding service code usually requires similar objects to be injected as in the handling of arriving messages. Therefore in grpc-scopes lib listenerEventScope⧉ scopes injections to the context of each single event callback (both from StreamObserver and ServerCallStreamObserver). (The listener part of the name comes from Listener ⧉ object associated with each RPC that invokes all these callbacks)

What if I Told You That Clients Also Need Scopes?

In the case of bi-di streaming methods, the distinction between the client and server sides becomes quite blurry: once a call is initiated, the server does not need to wait for any message from the client stream and may start sending his messages right away. The client may actually wait with his stream until the first message from the server arrives and then start sending messages that are actually responses to server messages. For example, workers may connect as gRPC clients to a manager acting as a gRPC server, to register and start receiving tasks to execute and then send back results. To process asynchronous messages from the server (manager), clients (workers) may need the injection of objects scoped to the context of a given task message from the server (manager).

Another, a more common situation is when a server, as a part of processing messages from clients, makes gRPC calls to another streaming server. For example, the first server may be a kind of proxy in front of the second server. Again, to process asynchronous responses coming from the second server, the first server may need the injection of objects scoped to the context of a given response message.

Therefore, both previously described listenerEventScope and rpcScope are available on the client side also: each callback that a client may receive will have a separate event context, and all callbacks related to some single given client RPC call will share the same RPC context.

How to Decide Which of These 2 Scopes Is Right for My Injection?

Very roughly speaking, if in a servlet app you would scope something with a @RequestScope, then often in a gRPC app, you should scope it with listenerEventScope. This is because request-scoped stuff usually needs to be short-lived or short-retained, as in the examples described before. However, request-scoped stuff that does not have this requirement may work better with rpcScope due to performance reasons as this reduces how often such stuff needs to be created/fetched.

As gRPC servers are by default stateless (there's no built-in mechanism to maintain a client state between separate RPCs), stuff scoped with @SessionScope usually ends up scoped with rpcScope in gRPC apps. If servlet-based REST services need to be ported to gRPC and maintaining an HttpSession was critical to their functionality, then a potential workaround is to translate REST calls to bi-di streaming calls, where 1 response message corresponds to 1 particular request message. This, however, requires clients to maintain their connections to the server for a long time, which in cases where clients are end-users is not feasible, especially in the case of users using mobile devices. In such cases, gRPC may basically not be a suitable solution.

Where Are @RpcScoped and @EventScoped Annotations?

grpc-scopes discourage overuse of annotations as they pollute code with hard-to-trace effects and instead promote defining injection bindings with plain old Java code using Guice Module⧉  objects. Furthermore, scoping annotations defeat the main purpose of dependency injection, which is decoupling component logic code from application wiring. Even worse, annotating classes with platform-specific annotations limits portability: for example, to reuse in a gRPC app components that are otherwise independent of Servlets or Spring, but were annotated with one of @RequestScoped/@SessionScoped/@RequestScope/@SessionScope, requires to include dependencies that don't serve any other purpose than providing these annotations, that are meaningless and confusing in gRPC context. As explained before, in Guice,every scope is an instance of a Scope class that can be used in a Module to define a scoped binding. For example:

bind(EntityManager.class)
		.toProvider(entityManagerFactory::createEntityManager)
		.in(grpcModule.listenerEventScope);

So Where Are the Static Vars With gRPC Scopes Similar to Those in Guice Servlet Extension?

grpc-scopes discourage the use of static context as it causes numerous issues. Instead, in the app's main method, a local instance of GrpcModule⧉ can be created that provides both scopes on its public fields. If, however, you cannot live without static Scope vars, then simply create a static instance of GrpcModule and copy both fields:

public class MyGrpcServer {
	public static final GrpcModule GRPC_MODULE = new GrpcModule();
	public static final Scope RPC_SCOPE = GRPC_MODULE.rpcScope;
	public static final Scope EVENT_SCOPE = GRPC_MODULE.listenerEventScope;

	public static void main(String[] args) {/* ...  */}

	// more code here...
}

How to Put It to Work?

1. Create an instance of GrpcModule as described above.
2. Create your other modules that may use scopes from the GrpcModule in their bindings as described before.
3. Create a Guice Injector by passing the above modules (and the GrpcModule).
4. Ask the above Injector for instances of your gRPC service classes and/or of your client response observer classes.
5. Use interceptors from GrpcModule as presented below:
6. Java

    

   grpcServer = ServerBuilder
   	.forPort(port)
   	.addService(ServerInterceptors.intercept(
   		myService, grpcModule.contextInterceptor /* more interceptors here... */))
   	// more services and other stuff here...
   	.build();

   For scopes to work in a server app, intercept services when adding them to a gRPC Server with GrpcModule.serverInterceptor as presented above.
   
   

7. Java

    

   final var managedChannel = ManagedChannelBuilder
   	.forTarget(TARGET)
   	.usePlaintext()
   	.build();
   final var channel = ClientInterceptors.intercept(
   		managedChannel, grpcModule.clientInterceptor);
   final var stub = MyServiceGrpc.newStub(channel);

   For scopes to work in a client app, before creating stubs, intercept Channel instances (like ManagedChannel) with GrpcModule.clientInterceptor as presented above.

That's it :) You can have a look at the project's README file. After that, you may have a look at a complete sample app that uses these scopes to properly inject JPA EntityManager instances.