Security

Setting up SSH access to a new VM usually follows the same routine: generate a key pair, copy it to the VM, tweak some configs, confirm the host’s identity, and maybe set up an agent to avoid typing passphrases all day. Tools like cloud-init and Ansible handle most of the setup for me now, so I rarely think about it. But I realized I don’t fully understand how all the parts work together.

A TOTP based 2FA system has two parts. One is a client that generates the TOTP code. The other part is a server. The server verifies the code. If the client and the server-generated codes match, the server allows the inbound user to access the target system. The code usually expires after 30 seconds and then, you’ll have to regenerate it to be able to authenticate.

As per RFC-6238, the server shares a base-32 encoded secret key with the client. Using this shared secret and the current UNIX timestamp, the client generates a 6-digit code. Independently, the server also generates a 6-digit code using the same secret string and its own current timestamp. If the user-entered client code matches the server-generated code, the auth succeeds. Otherwise, it fails. The client’s and the server’s current timestamp wouldn’t be an exact match. So the algorithm usually adjusts it for ~30 seconds duration.

While working with GitHub webhooks, I discovered a common webhook security pattern a receiver can adopt to verify that the incoming webhooks are indeed arriving from GitHub; not from some miscreant trying to carry out a man-in-the-middle attack. After some amount of digging, I found that it’s quite a common practice that many other webhook services employ as well. Also, check out how Sentry handles webhook verification.

Moreover, GitHub’s documentation demonstrates the pattern in Ruby. So I thought it’d be a good idea to translate that into Python in a more platform-agnostic manner. The core idea of the pattern goes as follows:

I was working on a DRF POST API endpoint where the consumer is expected to add a URL containing a PDF file and the system would then download the file and save it to an S3 bucket. While this sounds quite straightforward, there’s one big issue. Before I started working on it, the core logic looked like this:

# src.py
from __future__ import annoatations

from urllib.request import urlopen
import tempfile
from shutil import copyfileobj


def save_to_s3(src_url: str, dest_url: str) -> None:
    with tempfile.NamedTemporaryFile() as file:
        with urlopen(src_url) as response:
            # This stdlib function saves the content of the file
            # in 'file'.
            copyfileobj(response, file)

        # Logic to save file in s3.
        _save_to_s3(des_url)


if __name__ == "__main__":
    save_to_s3(
        "https://citeseerx.ist.psu.edu/viewdoc/download?"
        "doi=10.1.1.92.4846&rep=rep1&type=pdf",
        "https://s3-url.com",
    )

In the above snippet, there’s no guardrail against how large the target file can be. You could bring the entire server down to its knees by posting a link to a ginormous file. The server would be busy downloading the file and keep consuming resources.