API

Here’s a Python snippet that makes an HTTP POST request:

# script.py

import httpx
from typing import Any


async def make_request(url: str) -> dict[str, Any]:
    headers = {"Content-Type": "application/json"}

    async with httpx.AsyncClient(headers=headers) as client:
        response = await client.post(
            url,
            json={"key_1": "value_1", "key_2": "value_2"},
        )
        return response.json()

The function make_request makes an async HTTP request with the HTTPx library. Running this with asyncio.run(make_request("https://httpbin.org/post")) gives us the following output:

{
  "args": {},
  "data": "{\"key_1\": \"value_1\", \"key_2\": \"value_2\"}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Content-Length": "40",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "python-httpx/0.27.2",
    "X-Amzn-Trace-Id": "Root=1-66d5f7b0-2ed0ddc57241f0960f28bc91"
  },
  "json": {
    "key_1": "value_1",
    "key_2": "value_2"
  },
  "origin": "95.90.238.240",
  "url": "https://httpbin.org/post"
}

We’re only interested in the json field and want to assert in our test that making the HTTP call returns the expected values.

Let’s say you have a web app that emits log messages from different layers. Your log shipper collects and sends these messages to a destination like Datadog where you can query them. One common requirement is to tag the log messages with some common attributes, which you can use later to query them.

In distributed tracing, this tagging is usually known as context propagation, where you’re attaching some contextual information to your log messages that you can use later for query purposes. However, if you have to collect the context at each layer of your application and pass it manually to the downstream ones, that’d make the whole process quite painful.

People typically associate Google’s Protocol Buffer with gRPC services, and rightfully so. But things often get confusing when discussing protobufs because the term can mean different things:

• A binary protocol for efficiently serializing structured data.
• A language used to specify how this data should be structured.

In gRPC services, you usually use both: the protobuf language in proto files defines the service interface, and then the clients use the same proto files to communicate with the services.

One neat use case for the HTTP ETag header is client-side HTTP caching for GET requests. Along with the ETag header, the caching workflow requires you to fiddle with other conditional HTTP headers like If-Match or If-None-Match. However, their interaction can feel a bit confusing at times.

Every time I need to tackle this, I end up spending some time browsing through the relevant MDN docs on ETag, If-Match, and If-None-Match to jog my memory. At this point, I’ve done it enough times to justify spending the time to write this.

Ever since Rob Pike published the text on the functional options pattern, there’s been no shortage of blogs, talks, or comments on how it improves or obfuscates configuration ergonomics.

While the necessity of such a pattern is quite evident in a language that lacks default arguments in functions, more often than not, it needlessly complicates things. The situation gets worse if you have to maintain a public API where multiple configurations are controlled in this manner.

Suppose, you have a function that takes an option struct and a message as input. Then it stylizes the message according to the option fields and prints it. What’s the most sensible API you can offer for users to configure your function? Observe:

// app/src
package src

// Option struct
type Style struct {
    Fg string // ANSI escape codes for foreground color
    Bg string // Background color
}

// Display the message according to Style
func Display(s *Style, msg string) {}

In the src package, the function Display takes a pointer to a Style instance and a msg string as parameters. Then it decorates the msg and prints it according to the style specified in the option struct. In the wild, I’ve seen 3 main ways to write APIs that let users configure options:

I love Go’s implicit interfaces. While convenient, they can also introduce subtle bugs unless you’re careful. Types expected to conform to certain interfaces can fluidly add or remove methods. The compiler will only complain if an identifier anticipates an interface, but is passed a type that doesn’t implement that interface. This can be problematic if you need to export types that are required to implement specific interfaces as part of their API contract.

Recently, I purchased a domain for this blog and migrated the content from rednafi.github.io to rednafi.com. This turned out to be a much bigger hassle than I originally thought it’d be, mostly because, despite setting redirection for almost all the URLs from the previous domain to the new one and submitting the new sitemap.xml to the Search Console, Google kept indexing the older domain. To make things worse, the search engine selected the previous domain as canonical, and no amount of manual requests were changing the status in the last 30 days. Strangely, I didn’t encounter this issue with Bing, as it reindexed the new site within a week after I submitted the sitemap file via their webmaster panel.

While working with GitHub webhooks, I discovered a common webhook security pattern a receiver can adopt to verify that the incoming webhooks are indeed arriving from GitHub; not from some miscreant trying to carry out a man-in-the-middle attack. After some amount of digging, I found that it’s quite a common practice that many other webhook services employ as well. Also, check out how Sentry handles webhook verification.

Moreover, GitHub’s documentation demonstrates the pattern in Ruby. So I thought it’d be a good idea to translate that into Python in a more platform-agnostic manner. The core idea of the pattern goes as follows:

At my workplace, we have a large Django monolith that powers the main website and works as the primary REST API server at the same time. We use Django Rest Framework (DRF) to build and serve the API endpoints. This means, whenever there’s an error, based on the incoming request header - we’ve to return different formats of error responses to the website and API users.

The default DRF configuration returns a JSON response when the system experiences an HTTP 400 (bad request) error. However, the server returns an HTML error page to the API users whenever HTTP 403 (forbidden), HTTP 404 (not found), or HTTP 500 (internal server error) occurs. This is suboptimal; JSON APIs should never return HTML text whenever something goes wrong. On the other hand, the website needs those error text to appear accordingly.