Getting Started With Protobuf Using C# Runtime Library for Protocol Buffers

In this post, take a look at how we can quickly get started with Protobuf using C# runtime library for Protocol Buffers with only a few steps.

Kapil Khandelwal

Aug. 20, 21 · Tutorial

Likes (2)

Comment

Save

18.0K Views

In one of our client projects, we were using JSON as a default message format for data interchange between various distributed services. We were sending out large amounts and frequently updated data among our distributed services. So, we were evaluating other formats (eg. MessagePack, Protobuf, BSON) to compact message size and improve the performance. After completing our evaluation, we finalized Protobuf as our default message format.

Protobuf is the binary format crafted by Google which surpasses JSON performance (i.e. it is smaller, faster, and simpler). It is a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

In this post, I will be explaining how we can quickly get started with Protobuf using C# runtime library for Protocol Buffers with only a few steps.

Step 1: Install Google.Protobuf and Google.Protobuf.Tools NuGet Packages

Google.Protobuf: C# runtime library for Protocol Buffers

Google.Protobuf.Tools: Tools for Protocol Buffers

Step 2: Define Message Formats in a .proto File: Message Definitions (Schema)

For this post, I have created a message format for a typical Order Info class.

    ProtoBuf
   
 

   // Declares the syntax version being used (Version 3)
syntax = "proto3";

// Specifies the namespace to be used for the generated C# types. 
option csharp_namespace = "SampleApplication.ProtoBuf.Model";

//  Google's "Well Known Types" extensions: DateTime
import "google/protobuf/timestamp.proto"; 

//  Google's "Well Known Types" extensions: Nullable types
import "google/protobuf/wrappers.proto"; 

message OrderInfo {

  message DESTINATION {
    string name = 1;
  }

  DESTINATION destination = 1;

  message ORDERDATA {
    string sourceOrderId = 1;

    message ITEMS {
      string sku = 1;
      string sourceItemId = 2;

      message COMPONENTS {
        string code = 1;
        bool fetch = 2;
        string path = 3;
      }

      repeated COMPONENTS components = 3;
    }

    repeated ITEMS items = 2;

    message SHIPMENTS {

      message SHIPTO {
        string name = 1;
        string companyName = 2;
        string address1 = 3;
        string town = 4;
        string postcode = 5;
        string isoCountry = 6;
      }

      SHIPTO shipTo = 1;

      message CARRIER {
        string code = 1;
        string service = 2;
      }

      CARRIER carrier = 2;
    }

    repeated SHIPMENTS shipments = 3;
    
    // DateTime Type
    google.protobuf.Timestamp submittedDateTime = 4;
    uint32 orderTotal = 5;

    // Nullable Type uint?
    google.protobuf.UInt32Value orderDiscountAmount = 6;
  }

  ORDERDATA orderData = 2;
}
  

Note:

For supporting Nullable Types:

import "google/protobuf/wrappers.proto";

And then you can use:

google.protobuf.UInt32Value orderDiscountAmount = 6;

For supporting Dates & Times:

import "google/protobuf/timestamp.proto";

And then you can use:

google.protobuf.Timestamp submittedDateTime = 4;

Step 3: Use the Protoc Protocol Buffer Compiler To Generate C# Classes

This step is needed to read and write messages.

protoc -I=$SRC_DIR --csharp_out=$DST_DIR $FILEPATH

$SRC_DIR = Source directory (where your application's source code lives)
$DST_DIR = Destination directory (where you want the generated code to go)
$FILEPATH = Path of .proto file

For location of protoc complier:

Generate C# classes from OrderInfo.proto using protoc compliler.

Step 4, Part 1: Serialization

    C#
   
 

   private static byte[] SerializeOrderInfo(OrderInfo orderInfo)
{
  byte[] protoBufMessageBytes;

  using (var stream = new MemoryStream())
  {
    // Save the Order Info to a stream
    orderInfo.WriteTo(stream);
    protoBufMessageBytes = stream.ToArray();
  }

  return protoBufMessageBytes;
}
  

Step 4, Part 2: Deserialization

    C#
   
   private static OrderInfo DeserializeOrderInfo(byte[] protoBufMessageBytes) 
            => OrderInfo.Parser.ParseFrom(protoBufMessageBytes);

Step 5: Protobuf to JSON

For this step, use JSON Formatter (Reflection-based converter from messages to JSON).

    C#
   
 

   private static string OrderInfoToJson(OrderInfo orderInfo)
{
  var jsonFormatter = JsonFormatter.Default;
  string orderInfoJson = jsonFormatter.Format(orderInfo);
  return orderInfoJson;
}
  

Step 6: JSON to Protobuf

Step 6 uses JSON Parser (Reflection-based converter from JSON to messages).

    C#
   
   private static OrderInfo OrderInfoFromJson(string orderInfoJson) 
            => OrderInfo.Parser.ParseJson(orderInfoJson);

Sample Order Info Fill Method:

    C#
   
 

   private static OrderInfo FillOrderInfo()
{
  var orderInfo = new OrderInfo
  {
    Destination = new OrderInfo.Types.DESTINATION
    {
      Name = "accountName"
    },
    OrderData = new OrderInfo.Types.ORDERDATA
    {
      OrderTotal = 98765,
      SubmittedDateTime = Google.Protobuf.WellKnownTypes.Timestamp.FromDateTime(DateTime.UtcNow),
      SourceOrderId = "1234512345",
      OrderDiscountAmount = null,
    }
  };

  var item = new OrderInfo.Types.ORDERDATA.Types.ITEMS
  {
    Sku = "Business Cards",
    SourceItemId = "1234512346",
  };

  var component = new OrderInfo.Types.ORDERDATA.Types.ITEMS.Types.COMPONENTS
  {
    Code = "Content",
    Fetch = true,
    Path = "http://www.example.com/businessCard.pdf"
  };

  item.Components.Add(component);

  orderInfo.OrderData.Items.Add(item);

  var shipments = new OrderInfo.Types.ORDERDATA.Types.SHIPMENTS
  {
    Carrier = new OrderInfo.Types.ORDERDATA.Types.SHIPMENTS.Types.CARRIER
    {
      Code = "fedex",
      Service = "ground"
    },
    ShipTo = new OrderInfo.Types.ORDERDATA.Types.SHIPMENTS.Types.SHIPTO
    {
      Address1 = "1234 Main St.",
      CompanyName = "Acme",
      IsoCountry = "US",
      Name = "John Doe",
      Postcode = "12345",
      Town = "Capitol",
    }
  };

  orderInfo.OrderData.Shipments.Add(shipments);
  return orderInfo;
}
  

Sample Order Info JSON:

    JSON
   
 

   {
  "destination": {
    "name": "accountName"
  },
  "orderData": {
    "sourceOrderId": "1234512345",
    "items": [
      {
        "sku": "Business Cards",
        "sourceItemId": "1234512346",
        "components": [
          {
            "code": "Content",
            "fetch": true,
            "path": "http://www.example.com/businessCard.pdf"
          }
        ]
      }
    ],
    "shipments": [
      {
        "shipTo": {
          "name": "John Doe",
          "companyName": "Acme",
          "address1": "1234 Main St.",
          "town": "Capitol",
          "postcode": "12345",
          "isoCountry": "US"
        },
        "carrier": {
          "code": "fedex",
          "service": "ground"
        }
      }
    ],
    "submittedDateTime": "2021-07-08T11:09:08.727181600Z",
    "orderTotal": 98765
  }
}
  

That's it!

Protocol Buffers Protocol (object-oriented programming) Buffer (application) Library file IO

Published at DZone with permission of Kapil Khandelwal, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

Trending