Distributed Services with Go, Travis Jeffery
I’ve recently read Distributed Services with Go as part of my morning reading ritual. I’ve been working on a new service with Go for our experimentation platform at Udemy. This service will be a Kubernetes sidecar. Our default technologies for services are Kotlin and SpringBoot at Udemy. But, Kotlin and SpringBoot aren’t good enough technologies to implement a Kubernetes sidecar solution that should be efficient in resource usage, bootstrap time, docker image size, etc. So, we decided to use Go. I’ve some limited Go experience from my side projects and open source contribution activities, but I haven’t written any production-ready Go service from scratch. I choose this book to fill my knowledge gap.
The book is a good journey while implementing a simple distributed storage service from scratch; it’s a logging service like Kafka. It saves produced records to the file system and lets you consume them as tracking your offsets.
The concepts and technologies that you’ll use while reading the book;
- Protobuf and gRPC; API definitions and communication with RPC.
- SSL/TLS, mutual-TLS for encrypted data transfer, authentication, and authorization; securing the service from a zero-trust security perspective. You’ll use open source Cloudflare tools,
- Observability; Metric, Trace, and Log. You’ll use OpenCensus for metrics and traces, Uber Zap for structured logging,
- Server-to-Server service discovery makes your distributed service instances aware of each other; it’s for building and managing a cluster membership. You’ll use HashiCorp Serf.
- Consensus; your distributed service will be in a leader-follower structure. The leader will accept writes and replicate them to followers, which are used for reading. You’ll use HashiCorp Raft implementation.
- Load-balancing; You’ll use custom gRPC client-side load balancing. Clients will discover server instances and know which is the leader and followers.
- Deployment; You’ll deploy your service to a local Kubernetes cluster. You’ll use Kind(Kubernetes in Docker), Kubernetes StatefulSets, and Helm.
- Cloud; You’ll deploy your service to Google Cloud and make it publicly available.
The above list is exciting to see how different technologies are in action to build a distributed service from scratch. But as you guess, most of the above technologies deserve a separate book. You won’t find much detail in the book; the book primarily focuses on the practical use-cases rather than how these technologies work.
I learned many good things;
- mutual-TLS became more evident,
- I used the Uber Zap logging library in our service after seeing it in the book,
- I’ve tried Serf before but combining it with Raft creates such enlightenment,
- I tried client-side load-balancing with Kotlin before but couldn’t make it work, so it was great to see the process with Go.
- Seeing how a distributed storage service(even a simple one) works in practice was excellent.
I generally find it hard to read books that include much code, and it was the case also for this book. The codes are in the GitHub repo, but I would like to copy-paste them from the text to see step-by-step implementation in my IDE, but copying codes from the PDF is annoying. It would help if you had some pre-knowledge about the used technologies or learned them separately to entirely understand what’s happening with that technology. However, I like the practical approach of the book.
If you have already used these technologies in-depth, no need to learn their Go implementations, and have familiarity with distributed systems, then you can skip this book. Otherwise, I can recommend the book. Happy reading!