Skip to content

Latest commit

 

History

History
59 lines (47 loc) · 2.26 KB

20200406H.md

File metadata and controls

59 lines (47 loc) · 2.26 KB

April 6, 2020 Hour

Reading: Distributed Services with Go

Author(s): Dawn Schanafelt, Katharine Dvorak

Section: Chapter 2. Structure Data with Protocol Buffers

Finished: Chapter 2

JSON vs Protocol Buffers

Protobufs guarantee type-safety, prevents schema-violations, enables fast-serialization, and offer backwards compatibility. JSON has the benefit of being human-readable, but protobufs force you to define data transport schemas more concretely, which should lead to fewer bugs.

Once the structures surrounding the data are created in the Protobuf DSL, they can be compiled to any language. The author mentions that his Go projects had an entire repo called structs where all his teams data models were stored. This gives us a clean, maintainable way of accessing models and managing changes. We also getting free versioning, plugins to manage "extended functionality", and 6x faster serialization over JSON.

Install

brew install protobuf protoc --version

We can put protobufs in an api/vX directory in our project, where X represents the version of the API. To bump the major version of protobuf messages, just create a new v(X+1) directory.

Example

type Record struct {  
    Value []byte json:"value"
    Offset uint64 json:"offset"

turns into this:

syntax = "proto3";
// The package name identifies 'Record' as a part of 'log' and is used as the 
// package name in the compiled file.
package log.v1;  
import "gogoproto/gogo.proto";
// These options tell the protoc compiler to create methods for marshaling,
// unmarshaling, and finding the size of messages.
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.sizer_all) = true;
// Notice how every message is numbered, which allows us to version fields.
// Fields are immutable, you can stop using old fields or add new ones.
message Record { 
	bytes value = 1; 
	uint64 offset = 2;
}

Go has 2 runtimes for protobuf, with the forked gogprotobuf being more feature-rich and popular.

Install with:

go get github.com/gogo/protobuf

Compile with:

protoc api/v1/*.proto \
	--gogo_out=Mgogoproto/gogo.proto=github.com/gogo/protobuf/proto:. \
	--proto_path=$(go list -f '{{ .Dir }}' -m github.com/gogo/protobuf) \
	--proto_path=.