I spent the evening watching this incredibly grokkable talk on event-driven services by James Eastham at NDC London 2024. Below is a cleaned-up version of my notes. I highly recommend watching the full talk if you’re interested before reading this distillation. The curse of tightly coupled microservices Microservices often start with HTTP-based request-response communication, which seems straightforward but quickly becomes a pain as systems grow. Coupling—where one service depends on another—creates a few issues. Take the order processing service in a fictional Plant-Based Pizza company. It has to talk to the pickup service, delivery service, kitchen, and loyalty point service. They’re all tied together, so if one fails, the whole system could go down. ...

While going through a script at work today, I came across Bash’s nameref feature. It uses declare -n ref="$1" to set up a variable that allows you to reference another variable by name—kind of like pass-by-reference in C. I’m pretty sure I’ve seen it before, but I probably just skimmed over it. As I dug into the man page1, I realized there’s a gap in my understanding of how variable references actually work in Bash—probably because I never gave it proper attention and just got by cobbling together scripts. ...

When I started writing here about five years ago, I made a promise to myself that I wouldn’t give in to the trend of starting a blog, adding one overly enthusiastic entry about the stack behind it, and then vanishing into the ether. I was somewhat successful at that and wanted to write something I can link to when people are curious about the machinery that drives this site. The good thing is that the tech stack is simple and has remained stable over the years since I’ve only made changes when absolutely necessary. ...

I always struggle with the syntax for redirecting multiple streams to another command or a file. LLMs do help, but beyond the most obvious cases, it takes a few prompts to get the syntax right. When I know exactly what I’m after, scanning a quick post is much faster than wrestling with a non-deterministic kraken. So, here’s a list of the redirection and piping syntax I use the most, with real examples. ...

Here’s a Python snippet that makes an HTTP POST request: # script.py import httpx from typing import Any async def make_request(url: str) -> dict[str, Any]: headers = {"Content-Type": "application/json"} async with httpx.AsyncClient(headers=headers) as client: response = await client.post( url, json={"key_1": "value_1", "key_2": "value_2"}, ) return response.json() The function make_request makes an async HTTP request with the httpx1 library. Running this with asyncio.run(make_request("https://httpbin.org/post")) gives us the following output: { "args": {}, "data": "{\"key_1\": \"value_1\", \"key_2\": \"value_2\"}", "files": {}, "form": {}, "headers": { "Accept": "*/*", "Accept-Encoding": "gzip, deflate", "Content-Length": "40", "Content-Type": "application/json", "Host": "httpbin.org", "User-Agent": "python-httpx/0.27.2", "X-Amzn-Trace-Id": "Root=1-66d5f7b0-2ed0ddc57241f0960f28bc91" }, "json": { "key_1": "value_1", "key_2": "value_2" }, "origin": "95.90.238.240", "url": "https://httpbin.org/post" } We’re only interested in the json field and want to assert in our test that making the HTTP call returns the expected values. ...

I love @pytest.mark.parametrize1—so much so that I sometimes shoehorn my tests to fit into it. But the default style of writing tests with parametrize can quickly turn into an unreadable mess as the test complexity grows. For example: import pytest from math import atan2 def polarify(x: float, y: float) -> tuple[float, float]: r = (x**2 + y**2) ** 0.5 theta = atan2(y, x) return r, theta @pytest.mark.parametrize( "x, y, expected", [ (0, 0, (0, 0)), (1, 0, (1, 0)), (0, 1, (1, 1.5707963267948966)), (1, 1, (2**0.5, 0.7853981633974483)), (-1, -1, (2**0.5, -2.356194490192345)), ], ) def test_polarify(x: float, y: float, expected: tuple[float, float]) -> None: # pytest.approx helps us ignore floating point discrepancies assert polarify(x, y) == pytest.approx(expected) The polarify function converts Cartesian coordinates to polar coordinates. We’re using @pytest.mark.parametrize in its standard form to test different conditions. ...

I learned this neat Bash trick today where you can make a raw HTTP request using the /dev/tcp file descriptor without using tools like curl or wget. This came in handy while writing a health check script that needed to make a TCP request to a service. The following script opens a TCP connection and makes a simple GET request to example.com: #!/bin/bash # Open a TCP connection to example.com on port 80 and assign file descriptor 3 # The exec command keeps /dev/fd/3 open throughout the lifetime of the script # 3<> enables bidirectional read-write exec 3<>/dev/tcp/example.com/80 # Send the HTTP GET request to the server # >& redirects stdout to /dev/fd/3 echo -e "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n" >&3 # Read and print the server's response # <& redirects the output of /dev/fd/3 to cat cat <&3 # Close the file descriptor, terminating the TCP connection exec 3>&- Running this will print the response from the site to your console. ...

Let’s say you have a web app that emits log messages from different layers. Your log shipper collects and sends these messages to a destination like Datadog where you can query them. One common requirement is to tag the log messages with some common attributes, which you can use later to query them. In distributed tracing, this tagging is usually known as context propagation1, where you’re attaching some contextual information to your log messages that you can use later for query purposes. However, if you have to collect the context at each layer of your application and pass it manually to the downstream ones, that’d make the whole process quite painful. ...

With the recent explosion of LLM tools, I often like to kill time fiddling with different LLM client libraries and SDKs in one-off scripts. Lately, I’ve noticed that some newer tools frequently mess up the logger settings, meddling with my application logs. While it’s less common in more seasoned libraries, I guess it’s worth rehashing why hijacking the root logger isn’t a good idea when writing libraries or other forms of reusable code. ...

TIL about the install command on *nix systems. A quick GitHub search for the term brought up a ton of matches1. I’m surprised I just found out about it now. Often, in shell scripts I need to: Create a directory hierarchy Copy a config or binary file to the new directory Set permissions on the file It usually looks like this: # Create the directory hierarchy. The -p flag creates the parent directories # if they don't exist mkdir -p ~/.config/app # Copy the current config to the newly created directory. Here, conf already # exists in the current folder cp conf ~/.config/app/conf # Set the file permission chmod 755 ~/.config/app/conf Turns out, the install command in GNU coreutils2 can do all that in one line: ...

I was working on the deployment pipeline for a service that launches an app in a dedicated VM using GitHub Actions. In the last step of the workflow, the CI SSHs into the VM and runs several commands using a here document1 in bash. The simplified version looks like this: # SSH into the remote machine and run a bunch of commands to deploy the service ssh $SSH_USER@$SSH_HOST <<EOF # Go to the work directory cd $WORK_DIR # Make a git pull git pull # Export environment variables required for the service to run export AUTH_TOKEN=$APP_AUTH_TOKEN # Start the service docker compose up -d --build EOF The fully working version can be found on GitHub2. ...

One of the reasons why I’m a big advocate of rebasing and cleaning up feature branches, even when the changes get squash-merged to the mainline, is that it makes the PR reviewer’s life a little easier. I’ve written about my rebase workflow before1 and learned a few new things from the Hacker News discussion2 around it. While there’s been no shortage of text on why and how to craft atomic commits3, I often find those discussions focus too much on VCS hygiene, and the main benefit gets lost in the minutiae. When working in a team setup, I’ve discovered that individual commits matter much less than the final change list. ...

People tend to get pretty passionate about Git workflows on different online forums. Some like to rebase, while others prefer to keep the disorganized records. Some dislike the extra merge commit, while others love to preserve all the historical artifacts. There’s merit to both sides of the discussion. That being said, I kind of like rebasing because I’m a messy committer who: Usually doesn’t care for keeping atomic commits1. Creates a lot of short commits with messages like “fix” or “wip”. Likes to clean up the untidy commits before sending the branch for peer review. Prefers a linear history over a forked one so that git log --oneline --graph tells a nice story. Git rebase allows me to squash my disordered commits into a neat little one, which bundles all the changes with passing tests and documentation. Sure, a similar result can be emulated using git merge --squash feat_branch or GitHub’s squash-merge feature, but to me, rebasing feels cleaner. Plus, over time, I’ve subconsciously picked up the tricks to work my way around rebase-related gotchas. ...

People typically associate Google’s Protocol Buffer1 with gRPC2 services, and rightfully so. But things often get confusing when discussing protobufs because the term can mean different things: A binary protocol for efficiently serializing structured data. A language used to specify how this data should be structured. In gRPC services, you usually use both: the protobuf language in proto files defines the service interface, and then the clients use the same proto files to communicate with the services. ...

The handful of times I’ve reached for typing.TypeGuard in Python, I’ve always been confused by its behavior and ended up ditching it with a # type: ignore comment. For the uninitiated, TypeGuard allows you to apply custom type narrowing1. For example, let’s say you have a function named pretty_print that accepts a few different types and prints them differently onto the console: from typing import assert_never def pretty_print(val: int | float | str) -> None: if isinstance(val, int): # assert_type(val, int) print(f"Integer: {val}") elif isinstance(val, float): # assert_type(val, float) print(f"Float: {val}") elif isinstance(val, str): # assert_type(val, str) print(f"String: {val}") else: assert_never(val) If you run it through mypy, in each branch, the type checker automatically narrows the type and knows exactly what the type of val is. You can test the narrowed type in each branch with the typing.assert_type function. ...

One neat use case for the HTTP ETag header is client-side HTTP caching for GET requests. Along with the ETag header, the caching workflow requires you to fiddle with other conditional HTTP headers like If-Match or If-None-Match. However, their interaction can feel a bit confusing at times. Every time I need to tackle this, I end up spending some time browsing through the relevant MDN docs123 to jog my memory. At this point, I’ve done it enough times to justify spending the time to write this. ...

Every once in a while, I find myself skimming through the MDN docs to jog my memory on how CORS1 works and which HTTP headers are associated with it. This is particularly true when a frontend app can’t talk to a backend service I manage due to a CORS error2. MDN’s CORS documentation is excellent but can be a bit verbose for someone just looking for a way to quickly troubleshoot and fix the issue at hand. ...

Ever since Rob Pike published the text on the functional options pattern1, there’s been no shortage of blogs, talks, or comments on how it improves or obfuscates configuration ergonomics. While the necessity of such a pattern is quite evident in a language that lacks default arguments in functions, more often than not, it needlessly complicates things. The situation gets worse if you have to maintain a public API where multiple configurations are controlled in this manner. ...

In 9th grade, when I first learned about Lenz’s Law1 in Physics class, I was fascinated by its implications. It states: The direction of an induced current will always oppose the motion causing it. In simpler terms, imagine you have a hoop and a magnet. If you move the magnet close to the hoop, the hoop generates a magnetic field that pushes the magnet away. Conversely, if you pull the magnet away, the hoop creates a magnetic field that pulls it back. This occurs because the hoop aims to prevent any change in the surrounding magnetic field. That’s Lenz’s Law: the hoop consistently acts to maintain the magnetic field’s status quo, reacting against the motion that’s the cause of the existence of the magnetic flux in the first place. Generators leverage this principle to convert mechanical motion into electrical energy. ...

These days, I don’t build hierarchical types through inheritance even when writing languages that support it. Type composition has replaced almost all of my use cases where I would’ve reached for inheritance before. I’ve written1 about how to escape the template pattern2 hellscape and replace that with strategy pattern3 in Python before. While by default, Go saves you from shooting yourself in the foot by disallowing inheritance, it wasn’t obvious to me how I could apply the strategy pattern to make things more composable and testable. ...