Eschewing black box API calls

I love dynamically typed languages as much as the next person. They let us make ergonomic API calls like this: import httpx # Sync call for simplicity r = httpx.get("https://dummyjson.com/products/1").json() print(r["id"], r["title"], r["description"]) or this: fetch("https://dummyjson.com/products/1") .then((res) => res.json()) .then((json) => console.log(json.id, json.type, json.description)); In both cases, running the snippets will return: 1 'iPhone 9' 'An apple mobile which is nothing like apple' Unless you’ve worked with a statically typed language that enforces more constraints, it’s hard to appreciate how incredibly convenient it is to be able to call and use an API endpoint without having to deal with types or knowing anything about its payload structure. You can treat the API response as a black box and deal with everything in runtime. ...

January 15, 2024

Rate limiting via Nginx

I needed to integrate rate limiting into a relatively small service that complements a monolith I was working on. My initial thought was to apply it at the application layer, as it seemed to be the simplest route. Plus, I didn’t want to muck around with load balancer configurations, and there’s no shortage of libraries that allow me to do this quickly in the app. However, this turned out to be a bad idea. In the event of a DDoS1 or thundering herd2 incident, even if the app rejects the influx of inbound requests, the app server workers still have to do a minimal amount of work. ...

January 6, 2024

Configuring options in Go

Suppose, you have a function that takes an option struct and a message as input. Then it stylizes the message according to the option fields and prints it. What’s the most sensible API you can offer for users to configure your function? Observe: // app/src package src // Option struct type Style struct { Fg string // ANSI escape codes for foreground color Bg string // Background color } // Display the message according to Style func Display(s *Style, msg string) {} In the src package, the function Display takes a pointer to a Style instance and a msg string as parameters. Then it decorates the msg and prints it according to the style specified in the option struct. In the wild, I’ve seen 3 main ways to write APIs that let users configure options: ...

September 5, 2023

Dummy load balancer in a single Go script

I was curious to see if I could prototype a simple load balancer in a single Go script. Go’s standard library and goroutines make this trivial. Here’s what the script needs to do: Spin up two backend servers that’ll handle the incoming requests. Run a reverse proxy load balancer in the foreground. The load balancer will accept client connections and round-robin them to one of the backend servers; balancing the inbound load. Once a backend responds, the load balancer will relay the response back to the client. For simplicity, we’ll only handle client’s GET requests. Obviously, this won’t have SSL termination, advanced balancing algorithms, or session persistence like you’d get with Nginx1 or Caddy2. The point is to understand the basic workflow and show how Go makes it easy to write this sort of stuff. ...

August 30, 2023

Limit goroutines with buffered channels

I was cobbling together a long-running Go script to send webhook messages to a system when some events occur. The initial script would continuously poll a Kafka topic for events and spawn new goroutines to make HTTP requests to the destination. This had two problems: It could create unlimited goroutines if many events arrived quickly It might overload the destination system by making many concurrent requests In Python, I’d use just asyncio.Semaphore to limit concurrency. I’ve previously written about this here1. Turns out, in Go, you could do the same with a buffered channel. Here’s how the naive version looks: ...

August 23, 2023

Writing a TOTP client in Go

A TOTP1 based 2FA system has two parts. One is a client that generates the TOTP code. The other part is a server. The server verifies the code. If the client and the server-generated codes match, the server allows the inbound user to access the target system. The code usually expires after 30 seconds and then, you’ll have to regenerate it to be able to authenticate. As per RFC-62382, the server shares a base-32 encoded secret key with the client. Using this shared secret and the current UNIX timestamp, the client generates a 6-digit code. Independently, the server also generates a 6-digit code using the same secret string and its own current timestamp. If the user-entered client code matches the server-generated code, the auth succeeds. Otherwise, it fails. The client’s and the server’s current timestamp wouldn’t be an exact match. So the algorithm usually adjusts it for ~30 seconds duration. ...

August 20, 2023

Interface guards in Go

I love Go’s implicit interfaces. While convenient, they can also introduce subtle bugs unless you’re careful. Types expected to conform to certain interfaces can fluidly add or remove methods. The compiler will only complain if an identifier anticipates an interface, but is passed a type that doesn’t implement that interface. This can be problematic if you need to export types that are required to implement specific interfaces as part of their API contract. ...

August 18, 2023

Go structured logging with slog

Before the release of version 1.21, you couldn’t set levels for your log messages in Go without either using third-party libraries or writing your own boilerplates. Coming from Python, I’ve always found this odd, considering that this capability has been in the Python standard library forever. However, it seems like the new log/slog subpackage in Go allows you to do that and a whole lot more. Apart from being able to add levels to log messages, slog also allows you to emit JSON-structured log messages and group them by certain attributes. The ability to do all this in-house is quite neat and I wanted to take it for a spin. The official documentation1 on this is on the terser side but still comprehensive. So, here, instead of repeating the same information, I wanted to write something for me that mainly highlights the most common cases. ...

August 10, 2023