Building a Simple gRPC Service in Go

This post starts with the basics of building a gRPC service in Go, including tooling, best practices, and design considerations.

Sriram Panyam

Praveen Gujar

Feb. 05, 24 · Tutorial

Likes (2)

Comment

Save

2.4K Views

Client-server communication is a fundamental part of modern software architecture. Clients (on various platforms — web, mobile, desktop, and even IoT devices) request functionality (data and views) that servers compute, generate, and serve. There have been several paradigms facilitating this: REST/Http, SOAP, XML-RPC, and others.

gRPC is a modern, open source, and highly performant remote procedure call (RPC) framework developed by Google enabling efficient communication in distributed systems. gRPC also uses an interface definition language (IDL) — protobuf — to define services, define methods, and messages, as well as serializing structure data between servers and clients. Protobuf as a data serialization format is powerful and efficient — especially compared to text-based formats (like JSON). This makes a great choice for applications that require high performance and scalability.

A major advantage gRPC confers is its ability to generate code for several clients and servers in several languages (Java, Python, Go, C++, JS/TS) and target various platforms and frameworks. This simplifies implementing and maintaining consistent APIs (via a source-of-truth IDL) in a language and platform-agnostic way. gRPC also offers features like streaming (one-way and bi-directional), flow control, and flexible middleware/interceptor mechanisms, making it a superb choice for real-time applications and microservice architectures.

What makes the gRPC shine is its plugin facility and ecosystem for extending it on several fronts. With plugins just, some of the things you can do are:

Generate server stubs to implement your service logic
Generate clients to talk to these servers
Target several languages (golang, python, typescript, etc.)
Even targeting several transport types (HTTP, TCP, etc.)

Here is an awesome list of curated plugins for the gRPC ecosystem. For example - you can even generate an HTTP proxy gateway along with its own OpenAPI spec for those still needing them to consume your APIs by using the appropriate plugin.

There are some disadvantages to using gRPC:

Being a relatively recent technology, it has some complexities in learning, setting up, and use. This may especially be true for developers coming from more traditional methodologies (like REST)
Browser support for gRPC may be limited. Even though web clients can be generated, opening up access to noncustom ports (hosting gRPC services) may not be feasible due to org security policies.

Despite this (we feel) its advantages outweigh the disadvantages. Improved tooling over time, an increase in familiarity, and a robust plugin ecosystem have all made gRPC a popular choice. The browser support limitation will be addressed in a future article in the series.

In this article, we will build a simple gRPC service to showcase common features and patterns. This will serve as a foundation for the upcoming guides in this series. Let us get started!

Motivating Example

Let us build a simple service for powering a group chat application (like WhatsApp, Zulip, Slack, Teams, etc). Clearly, the goal is not to displace any of the existing popular services but rather to demonstrate the various aspects of a robust service powering a popular application genre. Our chat service — named OneHub — is simple enough. It has:

Topics: A place where a group of related users (by team, project, or interest) can share messages to communicate with each other. It is very similar to (but also much simpler than) channels in Slack or Microsoft.
Messages: The message being sent in the topic by users.

(Kudos if you have noticed that the "User" is missing. For now, we will ignore logins/auth and treat users simply with an opaque user ID. This will simplify testing our service over a number of features without worrying about login mechanisms, etc. We will come to all things about Auth, User management, and even social features in another future article). This service is rudimentary yet provides enough scope to take it in several directions, which will be the topic of dedicated future posts.

Prerequisites

This tutorial assumes you have the following already installed:

golang (1.18+)
Install protoc. On OSX it is as simple as brew install protobuf
gRPC protoc tools for generating Go
- go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
- go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Optional

We won't go too much into building services outside Go, but just for fun, we will also generate some of the Python stubs to show how easy it all is, and if there is popular demand one day, there could be an extension to this series covering other languages in more detail.

gRPC protoc tools for generating Python
- pyenv virtualenv onehub
- pyenv activate onehub
- pip install grpcio
- pip install grpcio-tools

Setup Your Project

The code for this can already be found in the OneHub repo. The repo is organized by service, and branches are used as checkpoints aligned with the end of each part in this series for easier revisiting.

     Go 
   
 
 
   mkdir onehub
cd onehub
# Replace this with your own github repo path
go mod init github.com/panyam/onehub
mkdir -p protos/onehub/v1
touch protos/onehub/v1/models.proto
touch protos/onehub/v1/messages.proto
touch protos/onehub/v1/topics.proto

# Install dependencies
go get google.golang.org/grpc 
  

Note when creating your protos, it is good practice to have them versioned (v1 above).

There are several ways to organize your protos, for example (and not limited to):

One giant proto for the entire service encompassing all models and protos (e.g., onehub.proto)
All Foo entity-related models and services in foo.proto
All models in a single proto (models.proto) accompanied by services for entity Foo in foo.proto.

In this series, we are using the 3rd approach as it also allows us to share models across services while still separating the individual entity services cleanly.

Define Your Service

The .proto files are the starting point of a gRPC, so we can start there with some basic details:

Models

     Go 
   
 
 
   syntax = "proto3";
import "google/protobuf/timestamp.proto";
import "google/protobuf/struct.proto";

option go_package = "github.com/onehub/protos";
package onehub.v1;

// Topics where users would post a message
message Topic {
  google.protobuf.Timestamp created_at = 1;
  google.protobuf.Timestamp updated_at = 2;

  // ID of the topic
  string id = 3;

  // ID of the user that created this topic
  string creator_id = 4;
  
  // A unique name of the topic that users can use to connect to
  string name = 5;

  // IDs of users in this topic.   Right now no information about
  // their participation is kept.
  repeated string users = 6;
}

/**
 * An individual message in a topic
 */
message Message {
  /**
   * When the message was created on the server.
   */
  google.protobuf.Timestamp created_at = 1;

  /**
   * When the message or its body were last modified (if modifications are
   * possible).
   */
  google.protobuf.Timestamp updated_at = 2;

  /**
   * ID of the message guaranteed to be unique within a topic.
   * Set only by the server and cannot be modified.
   */
  string id = 3;

  /**
   * User sending this message.
   */
  string user_id = 4;

  /**
   * Topic the message is part of.  This is only set by the server
   * and cannot be modified.
   */
  string topic_id = 5;

  /**
   * Content type of the message. Can be like a ContentType http
   * header or something custom like shell/command
   */
  string content_type = 6;

  /**
   * A simple way to just send text.
   */
  string content_text = 7;

  // Raw contents for data stored locally as JSON
  // Note we can have a combination of text, url and data
  // to show different things in the View/UI
  google.protobuf.Struct content_data = 8;
} 
  

Topic Service

View full code.

     Go 
   
 
 
   syntax = "proto3";
import "google/protobuf/field_mask.proto";

option go_package = "github.com/onehub/protos";
package onehub.v1;

import "onehub/v1/models.proto";

/**
 * Service for operating on topics
 */
service TopicService {
  /**
   * Create a new sesssion
   */
  rpc CreateTopic(CreateTopicRequest) returns (CreateTopicResponse) {
  }

  /**
   * List all topics from a user.
   */
  rpc ListTopics(ListTopicsRequest) returns (ListTopicsResponse) { 
  }

  /**
   * Get a particular topic
   */
  rpc GetTopic(GetTopicRequest) returns (GetTopicResponse) { 
  }

  /**
   * Batch get multiple topics by ID
   */
  rpc GetTopics(GetTopicsRequest) returns (GetTopicsResponse) { 
  }

  /**
   * Delete a particular topic
   */
  rpc DeleteTopic(DeleteTopicRequest) returns (DeleteTopicResponse) { 
  }

  /**
   * Updates specific fields of a topic
   */
  rpc UpdateTopic(UpdateTopicRequest) returns (UpdateTopicResponse) {
  }
}

/**
 * Topic creation request object
 */
message CreateTopicRequest {
  /**
   * Topic being updated
   */
  Topic topic = 1;
}

/**
 * Response of an topic creation.
 */
message CreateTopicResponse {
  /**
   * Topic being created
   */
  Topic topic = 1;
}

/**
 * An topic search request.  For now only paginations params are provided.
 */
message ListTopicsRequest {
  /**
   * Instead of an offset an abstract  "page" key is provided that offers
   * an opaque "pointer" into some offset in a result set.
   */
  string page_key = 1;

  /**
   * Number of results to return.
   */
  int32 page_size = 2;
}

/**
 * Response of a topic search/listing.
 */
message ListTopicsResponse {
  /**
   * The list of topics found as part of this response.
   */
  repeated Topic topics = 1;

  /**
   * The key/pointer string that subsequent List requests should pass to
   * continue the pagination.
   */
  string next_page_key = 2;
}

/**
 * Request to get an topic.
 */
message GetTopicRequest {
  /**
   * ID of the topic to be fetched
   */
  string id = 1;
}

/**
 * Topic get response
 */
message GetTopicResponse {
  Topic topic = 1;
}

/**
 * Request to batch get topics
 */
message GetTopicsRequest {
  /**
   * IDs of the topic to be fetched
   */
  repeated string ids = 1;
}

/**
 * Topic batch-get response
 */
message GetTopicsResponse {
  map<string, Topic> topics = 1;
}

/**
 * Request to delete an topic.
 */
message DeleteTopicRequest {
  /**
   * ID of the topic to be deleted.
   */
  string id = 1;
}

/**
 * Topic deletion response
 */
message DeleteTopicResponse {
}

/**
 * The request for (partially) updating an Topic.
 */
message UpdateTopicRequest {
  /**
   * Topic being updated
   */
  Topic topic = 1;

  /**
   * Mask of fields being updated in this Topic to make partial changes.
   */
  google.protobuf.FieldMask update_mask = 2;

  /**
   * IDs of users to be added to this topic.
   */
  repeated string add_users = 3;

  /**
   * IDs of users to be removed from this topic.
   */
  repeated string remove_users = 4;
}

/**
 * The request for (partially) updating an Topic.
 */
message UpdateTopicResponse {
  /**
   * Topic being updated
   */
  Topic topic = 1;
}
 
  

Message Service

View the full code here.

     Go 
   
 
 
   syntax = "proto3";
import "google/protobuf/field_mask.proto";

option go_package = "github.com/onehub/protos";
package onehub.v1;

import "onehub/v1/models.proto";

/**
 * Service for operating on messages
 */
service MessageService {
  /**
   * Create a new sesssion
   */
  rpc CreateMessage(CreateMessageRequest) returns (CreateMessageResponse) {
  }

  /**
   * List all messages in a topic
   */
  rpc ListMessages(ListMessagesRequest) returns (ListMessagesResponse) { 
  }

  /**
   * Get a particular message
   */
  rpc GetMessage(GetMessageRequest) returns (GetMessageResponse) { 
  }

  /**
   * Batch get multiple messages by IDs
   */
  rpc GetMessages(GetMessagesRequest) returns (GetMessagesResponse) { 
  }

  /**
   * Delete a particular message
   */
  rpc DeleteMessage(DeleteMessageRequest) returns (DeleteMessageResponse) { 
  }

  /**
   * Update a message within a topic.
   */
  rpc UpdateMessage(UpdateMessageRequest) returns (UpdateMessageResponse) {
  }
}

/**
 * Message creation request object
 */
message CreateMessageRequest {
  /**
   * Message being updated
   */
  Message message = 1;
}

/**
 * Response of an message creation.
 */
message CreateMessageResponse {
  /**
   * Message being created
   */
  Message message = 1;
}

/**
 * A message listing request.  For now only paginations params are provided.
 */
message ListMessagesRequest {
  /**
   * Instead of an offset an abstract  "page" key is provided that offers
   * an opaque "pointer" into some offset in a result set.
   */
  string page_key = 1;

  /**
   * Number of results to return.
   */
  int32 page_size = 2;

  /**
   * Topic in which messages are to be listed.  Required.
   */
  string topic_id = 3;
}

/**
 * Response of a topic search/listing.
 */
message ListMessagesResponse {
  /**
   * The list of topics found as part of this response.
   */
  repeated Message messages = 1;

  /**
   * The key/pointer string that subsequent List requests should pass to
   * continue the pagination.
   */
  string next_page_key = 2;
}

/**
 * Request to get a single message.
 */
message GetMessageRequest {
  /**
   * ID of the topic to be fetched
   */
  string id = 1;
}

/**
 * Message get response
 */
message GetMessageResponse {
  Message message = 1;
}

/**
 * Request to batch get messages
 */
message GetMessagesRequest {
  /**
   * IDs of the messages to be fetched
   */
  repeated string ids = 1;
}

/**
 * Message batch-get response
 */
message GetMessagesResponse {
  map<string, Message> messages = 1;
}

/**
 * Request to delete an message
 */
message DeleteMessageRequest {
  /**
   * ID of the message to be deleted.
   */
  string id = 1;
}

/**
 * Message deletion response
 */
message DeleteMessageResponse {
}

message UpdateMessageRequest {
  // The message being updated.  The topic ID AND message ID fields *must*
  // be specified in this message object.  How other fields are used is
  // determined by the update_mask parameter enabling partial updates
  Message message = 1;

  // Indicates which fields are being updated
  // If the field_mask is *not* provided then we reject
  // a replace (as required by the standard convention) to prevent
  // full replace in error.  Instead an update_mask of "*" must be passed.
  google.protobuf.FieldMask update_mask = 3;

  // Any fields specified here will be "appended" to instead of being
  // replaced
  google.protobuf.FieldMask append_mask = 4;
}

message UpdateMessageResponse {
  // The updated message
  Message message = 1;
}
 
  

Note each entity was relegated to its own service — though this does not translate to a separate server (or even process). This is merely for convenience.

For the most part, resource-oriented designs have been adopted for the entities, their respective services, and methods. As a summary:

Entities (in models.proto) have an id field to denote their primary key/object ID
All entities have a created/updated timestamp, which is set in the create and update methods, respectively.
All services have the typical CRUD methods.
The methods (rpcs) in each service follow similar patterns for their CRUD methods, e.g.:
- FooService.Get => method(GetFooRequest) => GetFooResponse
- FooService.Delete => method(DeleteFooRequest) => DeleteFooResponse
- FooService.Create => method(CreateFooRequest) => CreateFooResponse
- FooService.Update => method(UpdateFooRequest) => UpdateFooResponse
FooServer.Create methods take a Foo instance and set the instances id, created_at, and updated_at fields
FooService.Update methods take a Foo instance along with an update_mask to highlight fields being changed and update the fields. Additionally, it also ignores the id method, so an id cannot be over-written.

The entities (and relationships) are very straightforward. Some (slightly) noteworthy aspects are:

Topics have a list of IDs representing the participants in the topic (we are not focussing on the scalability bottlenecks from a large number of users in a Topic yet).
Messages hold a reference to the Topic (via topic_id).
The Message is very simple and only supports text messages (along with a way to pass in extra information or slightly custom message types - via content_data).

Generate the Service Stubs and Clients

The protoc command-line tool ensures that server (stubs) and clients are generated from this basic definition.

The magic of protoc is that it does not generate anything on its own. Instead, it uses plugins for different "purposes" to generate custom artifacts. First, let us generate go artifacts:

     Go 
   
 
 
   SRC_DIR=<ABSOLUTE_PATH_OF_ONEHUB>
PROTO_DIR:=$SRC_DIR/protos
OUT_DIR:=$SRC_DIR/gen/go
protoc --go_out=$OUT_DIR --go_opt=paths=source_relative               \
       --go-grpc_out=$OUT_DIR --go-grpc_opt=paths=source_relative     \
       --proto_path=$(PROTO_DIR)                                      \
        $PROTO_DIR/onehub/v1/*.proto 
  

This is quite cumbersome, so we can add this into a Makefile and simply run make all to generate protos and build everything going forward.

Makefile

View the code here.

     Go 
   
 
 
   # Some vars to detemrine go locations etc
GOROOT=$(which go)
GOPATH=$(HOME)/go
GOBIN=$(GOPATH)/bin

# Evaluates the abs path of the directory where this Makefile resides
SRC_DIR:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))

# Where the protos exist
PROTO_DIR:=$(SRC_DIR)/protos

# where we want to generate server stubs, clients etc
OUT_DIR:=$(SRC_DIR)/gen/go

all: printenv goprotos

goprotos:
    echo "Generating GO bindings"
    rm -Rf $(OUT_DIR) && mkdir -p $(OUT_DIR)
    protoc --go_out=$(OUT_DIR) --go_opt=paths=source_relative              \
       --go-grpc_out=$(OUT_DIR) --go-grpc_opt=paths=source_relative        \
       --proto_path=$(PROTO_DIR)                                                                             \
      $(PROTO_DIR)/onehub/v1/*.proto

printenv:
    @echo MAKEFILE_LIST=$(MAKEFILE_LIST)
    @echo SRC_DIR=$(SRC_DIR)
    @echo PROTO_DIR=$(PROTO_DIR)
    @echo OUT_DIR=$(OUT_DIR)
    @echo GOROOT=$(GOROOT)
    @echo GOPATH=$(GOPATH)
    @echo GOBIN=$(GOBIN)
 
  

Now, all generated stubs can be found in the gen/go/onehub/v1 folder (because our protos folder hosted the service defs within onehub/v1).

Briefly, the following are created:

For every X.proto file a gen/go/onehub/v1/X.pb.go is created. This file contains the model definition of every "message" in the .proto file (e.g., Topic and Message).
For every Y.proto file that contains servicedefinitions a X_grpc.pb.go file is generated that contains:
- A server interface must be implemented (coming in the next section).
- For a service X, an interface called XService is generated where the methods are all the RPC methods stipulated in the Y.proto file.
- A client is generated that can talk to a running implementation of the XService interface (coming below).

Pretty powerful, isn't it? Now, let us look at actually implementing the services.

Implementing Your Service

Our services are very simple. They store the different instances in memory as a simple collection of elements added in the order in which they were created (we will look at using a real database in the next part) and serve them by querying and updating this collection. Since all the services have (mostly) similar implementations (CRUD), a base store object has been created to represent the in-memory collection, and the services simply use this store.

This (simple) base entity looks like:

Base Entity Store

View full code.

     Go 
   
 
 
   package services

import (
    "fmt"
    "log"
    "sort"
    "time"

    tspb "google.golang.org/protobuf/types/known/timestamppb"
)

type EntityStore[T any] struct {
    IDCount  int
    Entities map[string]*T

    // Getters/Setters for ID
    IDSetter func(entity *T, id string)
    IDGetter func(entity *T) string

    // Getters/Setters for created timestamp
    CreatedAtSetter func(entity *T, ts *tspb.Timestamp)
    CreatedAtGetter func(entity *T) *tspb.Timestamp

    // Getters/Setters for udpated timestamp
    UpdatedAtSetter func(entity *T, ts *tspb.Timestamp)
    UpdatedAtGetter func(entity *T) *tspb.Timestamp
}

func NewEntityStore[T any]() *EntityStore[T] {
    return &EntityStore[T]{
        Entities: make(map[string]*T),
    }
}

func (s *EntityStore[T]) Create(entity *T) *T {
    s.IDCount++
    newid := fmt.Sprintf("%d", s.IDCount)
    s.Entities[newid] = entity
    s.IDSetter(entity, newid)
    s.CreatedAtSetter(entity, tspb.New(time.Now()))
    s.UpdatedAtSetter(entity, tspb.New(time.Now()))
    return entity
}

func (s *EntityStore[T]) Get(id string) *T {
    if entity, ok := s.Entities[id]; ok {
        return entity
    }
    return nil
}

func (s *EntityStore[T]) BatchGet(ids []string) map[string]*T {
    out := make(map[string]*T)
    for _, id := range ids {
        if entity, ok := s.Entities[id]; ok {
            out[id] = entity
        }
    }
    return out
}

// Updates specific fields of an Entity
func (s *EntityStore[T]) Update(entity *T) *T {
    s.UpdatedAtSetter(entity, tspb.New(time.Now()))
    return entity
}

// Deletes an entity from our system.
func (s *EntityStore[T]) Delete(id string) bool {
    _, ok := s.Entities[id]
    if ok {
        delete(s.Entities, id)
    }
    return ok
}

// Finds and retrieves entity matching the particular criteria.
func (s *EntityStore[T]) List(ltfunc func(t1, t2 *T) bool, filterfunc func(t *T) bool) (out []*T) {
    log.Println("E: ", s.Entities)
    for _, ent := range s.Entities {
        if filterfunc == nil || filterfunc(ent) {
            out = append(out, ent)
        }
    }
    // Sort in reverse order of name
    sort.Slice(out, func(idx1, idx2 int) bool {
        ent1 := out[idx1]
        ent2 := out[idx2]
        return ltfunc(ent1, ent2)
    })
    return
}
 
  

Using this, the Topic service is now very simple:

Topic Service Implementation

View the full code.

     Go 
   
 
 
   package services

import (
    "context"
    "log"
    "strings"

    protos "github.com/panyam/onehub/gen/go/onehub/v1"
    tspb "google.golang.org/protobuf/types/known/timestamppb"
)

type TopicService struct {
    protos.UnimplementedTopicServiceServer
    *EntityStore[protos.Topic]
}

func NewTopicService(estore *EntityStore[protos.Topic]) *TopicService {
    if estore == nil {
        estore = NewEntityStore[protos.Topic]()
    }
    estore.IDSetter = func(topic *protos.Topic, id string) { topic.Id = id }
    estore.IDGetter = func(topic *protos.Topic) string { return topic.Id }

    estore.CreatedAtSetter = func(topic *protos.Topic, val *tspb.Timestamp) { topic.CreatedAt = val }
    estore.CreatedAtGetter = func(topic *protos.Topic) *tspb.Timestamp { return topic.CreatedAt }

    estore.UpdatedAtSetter = func(topic *protos.Topic, val *tspb.Timestamp) { topic.UpdatedAt = val }
    estore.UpdatedAtGetter = func(topic *protos.Topic) *tspb.Timestamp { return topic.UpdatedAt }

    return &TopicService{
        EntityStore: estore,
    }
}

// Create a new Topic
func (s *TopicService) CreateTopic(ctx context.Context, req *protos.CreateTopicRequest) (resp *protos.CreateTopicResponse, err error) {
    resp = &protos.CreateTopicResponse{}
    resp.Topic = s.EntityStore.Create(req.Topic)
    return
}

// Get a single topic by id
func (s *TopicService) GetTopic(ctx context.Context, req *protos.GetTopicRequest) (resp *protos.GetTopicResponse, err error) {
    log.Println("Getting Topic by ID: ", req.Id)
    resp = &protos.GetTopicResponse{
        Topic: s.EntityStore.Get(req.Id),
    }
    return
}

// Batch gets multiple topics.
func (s *TopicService) GetTopics(ctx context.Context, req *protos.GetTopicsRequest) (resp *protos.GetTopicsResponse, err error) {
    log.Println("BatchGet for IDs: ", req.Ids)
    resp = &protos.GetTopicsResponse{
        Topics: s.EntityStore.BatchGet(req.Ids),
    }
    return
}

// Updates specific fields of an Topic
func (s *TopicService) UpdateTopic(ctx context.Context, req *protos.UpdateTopicRequest) (resp *protos.UpdateTopicResponse, err error) {
    resp = &protos.UpdateTopicResponse{
        Topic: s.EntityStore.Update(req.Topic),
    }
    return
}

// Deletes an topic from our system.
func (s *TopicService) DeleteTopic(ctx context.Context, req *protos.DeleteTopicRequest) (resp *protos.DeleteTopicResponse, err error) {
    resp = &protos.DeleteTopicResponse{}
    s.EntityStore.Delete(req.Id)
    return
}

// Finds and retrieves topics matching the particular criteria.
func (s *TopicService) ListTopics(ctx context.Context, req *protos.ListTopicsRequest) (resp *protos.ListTopicsResponse, err error) {
    results := s.EntityStore.List(func(s1, s2 *protos.Topic) bool {
        return strings.Compare(s1.Name, s2.Name) < 0
    }, nil)
    log.Println("Found Topics: ", results)
    resp = &protos.ListTopicsResponse{Topics: results}
    return
}
 
  

The Message service is also eerily similar and can be found here.

Wrap It All With a Runner

We have implemented the services with our logic, but the services need to be brought up.

The general steps are:

Create a GRPC Server instance
Register each of our service implementations with this server
Run this server on a specific port

Main Server CLI

Full code here.

     Go 
   
 
 
   package main

import (
    "flag"
    "log"
    "net"

    "google.golang.org/grpc"

    v1 "github.com/panyam/onehub/gen/go/onehub/v1"
    svc "github.com/panyam/onehub/services"

    // This is needed to enable the use of the grpc_cli tool
    "google.golang.org/grpc/reflection"
)

var (
    addr = flag.String("addr", ":9000", "Address to start the onehub grpc server on.")
)

func startGRPCServer(addr string) {
    // create new gRPC server
    server := grpc.NewServer()
    v1.RegisterTopicServiceServer(server, svc.NewTopicService(nil))
    v1.RegisterMessageServiceServer(server, svc.NewMessageService(nil))
    if l, err := net.Listen("tcp", addr); err != nil {
        log.Fatalf("error in listening on port %s: %v", addr, err)
    } else {
        // the gRPC server
        log.Printf("Starting grpc endpoint on %s:", addr)
        reflection.Register(server)
        if err := server.Serve(l); err != nil {
            log.Fatal("unable to start server", err)
        }
    }
}

func main() {
    flag.Parse()
    startGRPCServer(*addr)
}
 
  

This server can now be run (by default on port 9000) with:

     Go 
   
   go run cmd/server.go

Note this is a simple service with Unary RPC methods. i.e., the client sends a single request to the server and waits for a single response. There are also other types of methods.

Server streaming RPC: The client sends a request to the server and receives a stream of responses (similar to long-polling in HTTP, where the client listens to chunks of messages on the open connection).
Client streaming RPC: Here, the client sends a stream of messages in a single request and receives a single response from the server. For example, a single request from the client could involve multiple location updates (spread out over time), and the response from the server could be a single "path" object the client traveled along.
Bidirectional streaming RPC: The client initiates a connection with the server, and both the client and server can send messages independently of one another. The similarity for this in the HTTP universe would be Websocket connections.

We will implement one or more of these in future tutorials.

Client Calls to the Server

Now, it is time to test our server. Note the grpc server is not a REST endpoint. So, curl would not work (we will cover this in the next part). We can make calls against the server in a couple of ways — using a CLI utility (much like curl for REST/HTTP services) or by using the clients generated by the `protocol` tool. Even better, we can also make client calls from other languages — if we had opted to generate libraries targeting those languages, too.

Calling the Server via `grpc_cli` Utility

A grpc client (grpc_cli) exists to make direct calls from the command line. On OSX, this can be installed with brew install grpc.

If the server is not running, then go ahead and start it (as per the previous section). We can now start calling operations on the server itself — either to make calls or reflect on it!

List All Operations

grpc_cli ls localhost:9000 -l

     Go 
   
 
 
   filename: grpc/reflection/v1/reflection.proto
package: grpc.reflection.v1;
service ServerReflection {
  rpc ServerReflectionInfo(stream grpc.reflection.v1.ServerReflectionRequest) returns (stream grpc.reflection.v1.ServerReflectionResponse) {}
}

filename: grpc/reflection/v1alpha/reflection.proto
package: grpc.reflection.v1alpha;
service ServerReflection {
  rpc ServerReflectionInfo(stream grpc.reflection.v1alpha.ServerReflectionRequest) returns (stream grpc.reflection.v1alpha.ServerReflectionResponse) {}
}

filename: onehub/v1/messages.proto
package: onehub.v1;
service MessageService {
  rpc CreateMessage(onehub.v1.CreateMessageRequest) returns (onehub.v1.CreateMessageResponse) {}
  rpc ListMessages(onehub.v1.ListMessagesRequest) returns (onehub.v1.ListMessagesResponse) {}
  rpc GetMessage(onehub.v1.GetMessageRequest) returns (onehub.v1.GetMessageResponse) {}
  rpc GetMessages(onehub.v1.GetMessagesRequest) returns (onehub.v1.GetMessagesResponse) {}
  rpc DeleteMessage(onehub.v1.DeleteMessageRequest) returns (onehub.v1.DeleteMessageResponse) {}
  rpc UpdateMessage(onehub.v1.UpdateMessageRequest) returns (onehub.v1.UpdateMessageResponse) {}
}

filename: onehub/v1/topics.proto
package: onehub.v1;
service TopicService {
  rpc CreateTopic(onehub.v1.CreateTopicRequest) returns (onehub.v1.CreateTopicResponse) {}
  rpc ListTopics(onehub.v1.ListTopicsRequest) returns (onehub.v1.ListTopicsResponse) {}
  rpc GetTopic(onehub.v1.GetTopicRequest) returns (onehub.v1.GetTopicResponse) {}
  rpc GetTopics(onehub.v1.GetTopicsRequest) returns (onehub.v1.GetTopicsResponse) {}
  rpc DeleteTopic(onehub.v1.DeleteTopicRequest) returns (onehub.v1.DeleteTopicResponse) {}
  rpc UpdateTopic(onehub.v1.UpdateTopicRequest) returns (onehub.v1.UpdateTopicResponse) {}
} 
  

Create a Topic

grpc_cli --json_input --json_output call localhost:9000 CreateTopic '{topic: {name: "First Topic", creator_id: "user1"}}'

     Go 
   
 
 
   {
 "topic": {
  "createdAt": "2023-07-28T07:30:54.633005Z",
  "updatedAt": "2023-07-28T07:30:54.633006Z",
  "id": "1",
  "creatorId": "user1",
  "name": "First Topic"
 }
} 
  

And another

grpc_cli --json_input --json_output call localhost:9000 CreateTopic '{topic: {name: "Urgent topic", creator_id: "user2", users: ["user1", "user2", "user3"]}}'

     Go 
   
 
 
   {
 "topic": {
  "createdAt": "2023-07-28T07:32:04.821800Z",
  "updatedAt": "2023-07-28T07:32:04.821801Z",
  "id": "2",
  "creatorId": "user2",
  "name": "Urgent topic",
  "users": [
   "user1",
   "user2",
   "user3"
  ]
 }
} 
  

List All Topics

grpc_cli --json_input --json_output call localhost:9000 ListTopics {}

     Go 
   
 
 
   {
 "topics": [
  {
   "createdAt": "2023-07-28T07:30:54.633005Z",
   "updatedAt": "2023-07-28T07:30:54.633006Z",
   "id": "1",
   "creatorId": "user1",
   "name": "First Topic"
  },
  {
   "createdAt": "2023-07-28T07:32:04.821800Z",
   "updatedAt": "2023-07-28T07:32:04.821801Z",
   "id": "2",
   "creatorId": "user2",
   "name": "Urgent topic",
   "users": [
    "user1",
    "user2",
    "user3"
   ]
  }
 ]
}
 
  

Get Topics by IDs

grpc_cli --json_input --json_output call localhost:9000 GetTopics '{"ids": ["1", "2"]}'

     Go 
   
 
 
   {
 "topics": {
  "1": {
   "createdAt": "2023-07-28T07:30:54.633005Z",
   "updatedAt": "2023-07-28T07:30:54.633006Z",
   "id": "1",
   "creatorId": "user1",
   "name": "First Topic"
  },
  "2": {
   "createdAt": "2023-07-28T07:32:04.821800Z",
   "updatedAt": "2023-07-28T07:32:04.821801Z",
   "id": "2",
   "creatorId": "user2",
   "name": "Urgent topic",
   "users": [
    "user1",
    "user2",
    "user3"
   ]
  }
 }
}
 
  

Delete a Topic Followed by a Listing

grpc_cli --json_input --json_output call localhost:9000 DeleteTopic '{"id": "1"}'

     Go 
   
   connecting to localhost:9000
{}
Rpc succeeded with OK status

grpc_cli --json_input --json_output call localhost:9000 ListTopics {}

     Go 
   
 
 
   {
 "topics": [
  {
   "createdAt": "2023-07-28T07:32:04.821800Z",
   "updatedAt": "2023-07-28T07:32:04.821801Z",
   "id": "2",
   "creatorId": "user2",
   "name": "Urgent topic",
   "users": [
    "user1",
    "user2",
    "user3"
   ]
  }
 ]
} 
  

Programmatically Calling the Server

Instead of going into this deep, the tests in the service folder show how clients can be created as well as how to write tests.

Conclusion

That was a lot to cover, but we made it. Even though it was a basic example, (hopefully) it set a good foundation for the topics in the rest of the series.

In summary, gRPC is a crucial component of modern software development that allows developers to build high-performance, scalable, and interoperable systems. Its features and benefits have made it a popular choice among companies that need to handle large amounts of data or need to support real-time communication across multiple platforms.

In this article, we:

Created a grpc service in Go from the ground up, with a very simple implementation (sadly lacking in persistence),
Exercised the CLI utility to make calls to the running service
Exercised the generated clients while also writing tests

In this article we created a lot of the foundation from first principles for the purpose of demystification. In future articles, we will build upon these and look at more advanced tooling and topics like:

Rest/HTTP gateways for API access via HTTP
Dockerizing our environments for easy deployments, testing, and scaling
Persistence with a *real* database
A minimalistic fanfare-less web UI
Database replication strategies to power search indexes (with reliability and consistency in mind)
Scaling strategies
Converting into a pure multi-tenanted SaaS offering
And more...

gRPC Go (programming language) Framework microservice

Published at DZone with permission of Sriram Panyam. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending