Building a Go Api: gRPC, Rest and OpenApi (swagger)

mch1307

6 minute read

gRPC is an open source RPC framework offering high performance and pluggable support for authentication, tracing, health checks and load balancing. It offers libraries in most widely used languages (Java, Node.js, C++, Python, Go,..)

In this post, we will create a pseudo “Home control” server that will expose some APIs using gRPC. We will then add a Rest API using grpc-gateway and generate an OpenAPI documentation.

Prerequisites

The following components should be installed:

  1. Golang v1.5+

  2. gRPC:

    go get google.golang.org/grpc

  3. Protocol Buffers 3 (https://github.com/google/protobuf/releases)

  4. Go protoc plugin:

      go get -u github.com/golang/protobuf/protoc-gen-go`

  5. grpc-gateway

      go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
  go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
  go get -u github.com/golang/protobuf/protoc-gen-go

Consult the gRPC getting started guide for further details.

Project

In this example, we will develop a pseudo home control system. We will define a few devices with basic properties like name, hardware, location, type and status. We want to be able to list all devices, find a device by it’s ID, switch a device on or off and register a new device. We will populate the system with a few items from a json file.

This is how we would represent a device in a Go struct:

type device struct {
  ID       int    `json:"id"`
  Hardware string `json:"hardware"`
  Name     string `json:"name"`
  Location string `json:"location"`
  Type     string `json:"type"`
  Unit     string `json:"unit"`
  State    int    `json:"state"`
}

We will proceed to the following steps to build our project:

  • define the Protocol Buffers service, methods and message types
  • generate the Go server and client code
  • implement the generated server and client code into our app
  • add the grpc-gateway to provide Rest API and OpenAPI doc

Project sources are available on GitHub

Create the .proto file

gRPC uses Google’s Protocol Buffers by default. We will use basic RPCs (unary) but it offers other options like streams,.. You can get more about protobuf and gRPC concepts on grpc.io.

The goal is to define the Service, it’s methods and their input/output messages types.We will use the proto3 spec. Full language guide is available here.

service DeviceService {
  // List all registered devices
  rpc GetAllDevices(Empty) returns (Devices) {}
  // Get a device by ID
  rpc GetDeviceByID(ID) returns (Device) {}
  // Update a device’s state
  rpc SwitchDevice(UpdateDevice) returns (Device) {}
  // Register a new device
  rpc RegisterDevice(Device) returns (Device) {}
}

We have defined a “Device” service having 4 methods. Let’s define the different message types those methods are using:

message ID {
  int32 id = 1;
};

message UpdateDevice {
  int32 id = 1;
  int32 value = 2;
};

message Device {
  int32 id = 1;
  string hardware = 2;
  string name = 3;
  string location = 4;
  enum DeviceType {
  onOff = 0;
  dimmer = 1;
  sensor = 2;
}
      
DeviceType type = 5;
  string unit = 6;
  int32 state = 7;
};

message Devices {
  repeated Device device = 1;
};

  message Empty {
};

The Empty message is just a way to define a method that takes no argument.

Save this to a file called device.proto in a subfolder. I have named the folder pb.

Generate the Go client and server code

The protoc executable will do the job. Open a terminal and cd to the place you saved your proto file and type:

protoc --go out=plugins=grpc:. device.proto

This will generate a device.pb.go file containing server and client code.

Develop the server app

We now need to implement the desired functionalities. In our example, this will be mainly in a package called “db”.

Once the functionalities are there, we need to implement the gRPC part that has been previously generated by protoc:

type DeviceService struct{}

func (s *DeviceService) GetAllDevices(ctx context.Context, req *pb.Empty) (*pb.Devices, error) {
  devices := db.GetAllDevices()
  return &devices, nil
}
func (s *DeviceService) GetDeviceByID(ctx context.Context, id *pb.ID) (*pb.Device, error) {
  device := db.GetDeviceByID(id.Id)
  return device, nil
}
func (s *DeviceService) SwitchDevice(ctx context.Context, device *pb.UpdateDevice) (*pb.Device, error) {
  updatedDevice, err := db.SwitchDevice(device.Id, device.Value)
  if err != nil {
    log.Println("error updating device ", err)
  }
return updatedDevice, err
}

As a last step, we need to start our grpc server:

grpcPort := "8082"
// start listening for grpc
listen, err := net.Listen("tcp", grpcPort)
if err != nil {
  log.Fatal(err)
}
// Create new grpc server
server := grpc.NewServer()
// Register service
pb.RegisterDeviceServiceServer(server, new(DeviceService))
// Start serving requests
server.Server(listen)

We are now able to server gRPC requests on port 8082

Develop a client app

Now it’s time to try our gRPC server. We will develop a small client app that will get the list of devices from the gRPC server.

package main
 
import (
  "context"
  "fmt"
  "github.com/mch1307/go-ws-api/pb"
  "google.golang.org/grpc"
)
var empty pb.Empty
func main() {
  serverAddr := "localhost:8082"
  conn, err := grpc.Dial(serverAddr, grpc.WithInsecure())
  if err != nil {
    fmt.Println("error connecting: ", err)
  }
defer conn.Close()
client := pb.NewDeviceServiceClient(conn)
devices, err := client.GetAllDevices(context.Background(), &empty)
if err != nil {
  fmt.Println("error in grpc call:", err)
}
for _, dev := range devices.Device {
  fmt.Println(dev)
  }
}

Try our client app

go run cli.go
id:1 hardware:"philips" name:"light" location:"kitchen" type:onOff state:100
id:2 hardware:"osram" name:"light" location:"bedroom" type:onOff state:100
id:3 hardware:"artemide" name:"kitchen light" location:"living room" type:dimmer state:30
id:4 hardware:"oregon scientific" name:"temperature" location:"bedroom" type:sensor state:2
id:5 hardware:"oregon scientific" name:"humidity" location:"bedroom" type:sensor state:40
id:6 hardware:"osram" name:"light" location:"bedroom 2" type:onOff

We now have a working gRPC server and its client, but we would like to offer the same API through Rest/json.

Adding Rest

We will use grpc-gateway that will act as a proxy between a Rest client and our gRPC server. It will publish the endpoints based on our proto file. We also have the possibility to generate a Swagger/OpenAPI2 json file to document our Rest API.

The first step is to enrich our proto file with some data (annotations) so that protoc is able to generate the required code:

syntax= "proto3";
package pb;
import "google/api/annotations.proto";
service DeviceService {
// List all registered devices
rpc.GetAllDevices(Empty) returns (Devices){
option (google.api.http) = {
get: "/api/v1/devices"
};

At line 3 we import annotations

Then at line 8 and 9, we link the GetAllDevices method to what will become our Rest endpoint: GET on /api/v1/devices

We repeat this for all methods and once done, we can generate our Rest gateway using a command like the following:

protoc -I. -I%GOPATH%\src -I%GOPATH%\src\github.com\grpc-ecosystem\grpc-gateway\third_party\googleapis –grpc-gateway_out=logtostderr=true:. device.proto

Generate the OpenAPI doc

To complete our work, we need to generate an OpenAPI (formerly Swagger) doc. grp-gateway can also do that, based on annotations from out proto file. It will use the annotations we added previously for generating the proxy. We can add more annotations to generate a complete OpenAPI spec:

option (grpc.gateway.protoc\_gen\_swagger.options.openapiv2_swagger) = {
  info: {
  title: "go-ws-api";
  version: "1.0";
  contact: {
  name: "go-ws-api";
  url: "https://github.com/mch1307/go-ws-api";
  email: "none@example.com";
  };
};
  
schemes: HTTP;
  
schemes: HTTPS;
  
consumes: "application/json";
  
produces: "application/json";
  
};

We can now invoke protoc with the following arguments to generate our OpenAPI json file:

protoc -I. -I%GOPATH%\src -I%GOPATH%\src\github.com\grpc-ecosystem\grpc-gateway\third\_party\googleapis –swagger\_out=logtostderr=true:. device.proto

This is what the generated file looks like:

Conclusion

This was a simple project example and using gRPC is this case might be overkill. But it shows how grpc-gateway can be used on top of gRPC in order to offer both a highly efficient API with gRPC and wxpose it as a RESTful API.

We only used very basic gRPC functionalities, if you want to discover more of its numerous possibilities and its Go ecosystem, I suggest you read this article.

There is also another tool to use gRPC with Go that is worth checking: proteus from source{d}. The approach is to scan your Go code and generate a proto file as well as idiomatic Go server and client code. I haven’t try it yet but it could be rhe subject of a future post.

comments powered by Disqus