Python

The functools.wraps decorator allows you to keep your function’s identity intact after it’s been wrapped by a decorator. Whenever a function is wrapped by a decorator, identity properties like - function name, docstring, annotations of it get replaced by those of the wrapper function. Consider this example:

from __future__ import annotations

# In < Python 3.9, import this from the typing module.
from collections.abc import Callable
from typing import Any


def log(func: Callable) -> Callable:
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        """Internal wrapper."""

        val = func(*args, **kwargs)
        return val

    return wrapper


@log
def add(x: int, y: int) -> int:
    """Add two numbers.

    Parameters
    ----------
    x : int
        First argument.
    y : int
        Second argument.

    Returns
    -------
    int
        Returns the summation of two integers.
    """
    return x + y


if __name__ == "__main__":
    print(add.__doc__)
    print(add.__name__)

Here, I’ve defined a simple logging decorator that wraps the add function. The function add has its own type annotations and docstring. So, you’d expect the docstring and name of the add function to be printed when the above snippet gets executed. However, running the script prints the following instead:

I was working with a rate-limited API endpoint where I continuously needed to send short-polling GET requests without hitting HTTP 429 error. Perusing the API doc, I found out that the API endpoint only allows a maximum of 100 requests per second. So, my goal was to find out a way to send the maximum amount of requests without encountering the too-many-requests error.

I picked up Python’s asyncio and the amazing HTTPx library by Tom Christie to make the requests. This is the naive version that I wrote in the beginning; it quickly hits the HTTP 429 error:

Whether you like it or not, the split world of sync and async functions in the Python ecosystem is something we’ll have to live with; at least for now. So, having to write things that work with both sync and async code is an inevitable part of the journey. Projects like Starlette, HTTPx can give you some clever pointers on how to craft APIs that are compatible with both sync and async code.

While grokking Black formatter’s codebase, I came across this Rust-influenced error handling model that offers an interesting way of handling exceptions in Python. Exception handling in Python usually follows the EAFP paradigm where it’s easier to ask for forgiveness than permission.

However, Rust has this recoverable error handling workflow that leverages generic Enums. I wanted to explore how Black emulates that in Python. This is how it works:

# src.py
from __future__ import annotations

from typing import Generic, TypeVar, Union

T = TypeVar("T")
E = TypeVar("E", bound=Exception)


class Ok(Generic[T]):
    def __init__(self, value: T) -> None:
        self._value = value

    def ok(self) -> T:
        return self._value


class Err(Generic[E]):
    def __init__(self, e: E) -> None:
        self._e = e

    def err(self) -> E:
        return self._e


Result = Union[Ok[T], Err[E]]

In the above snippet, two generic types Ok and Err represent the return type and the error types of a callable respectively. These two generics were then combined into one Result generic type. You’d use the Result generic to handle exceptions as follows:

I’ve always had a hard time explaining variance of generic types while working with type annotations in Python. This is an attempt to distill the things I’ve picked up on type variance while going through PEP-483.

A pinch of type theory

A generic type is a class or interface that is parameterized over types. Variance refers to how subtyping between the generic types relates to subtyping between their parameters' types.

How’d you create a sub dictionary from a dictionary where the keys of the sub-dict are provided as a list?

I was reading a tweet by Ned Bachelder on this today and that made me realize that I usually solve it with O(DK) complexity, where K is the length of the sub-dict keys and D is the length of the primary dict. Here’s how I usually do that without giving it any thoughts or whatsoever:

I was reading a tweet about it yesterday and that didn’t stop me from pushing a code change in production with the same rookie mistake today. Consider this function:

# src.py
from __future__ import annotations

import logging
import time
from datetime import datetime


def log(
    message: str,
    /,
    *,
    level: str,
    timestamp: str = datetime.utcnow().isoformat(),
) -> None:
    logger = getattr(logging, level)

    # Avoid f-string in logging as it's not lazy.
    logger("Timestamp: %s \nMessage: %s\n" % (timestamp, message))


if __name__ == "__main__":
    for _ in range(3):
        time.sleep(1)
        log("Reality can often be disappointing.", level="warning")

Here, the function log has a parameter timestamp that computes its default value using the built-in datetime.utcnow().isoformat() method. I was under the impression that the timestamp parameter would be computed each time when the log function was called. However, that’s not what happens when you try to run it. If you run the above snippet, you’ll get this instead:

I used to use Unittest’s self.assertTrue / self.assertFalse to check both literal booleans and truthy/falsy values in Unittest. Committed the same sin while writing tests in Django.

I feel like assertTrue and assertFalse are misnomers. They don’t specifically check literal booleans, only truthy and falsy states respectively.

Consider this example:

# src.py
import unittest


class TestFoo(unittest.TestCase):
    def setUp(self):
        self.true_literal = True
        self.false_literal = False
        self.truthy = [True]
        self.falsy = []

    def is_true(self):
        self.assertTrue(self.true_literal, True)

    def is_false(self):
        self.assertFalse(self.false_literal, True)

    def is_truthy(self):
        self.assertTrue(self.truthy, True)

    def is_falsy(self):
        self.assertFalse(self.falsy, True)


if __name__ == "__main__":
    unittest.main()

In the above snippet, I’ve used assertTrue and assertFalse to check both literal booleans and truthy/falsy values. However, to test the literal boolean values, assertIs works better and is more explicit. Here’s how to do the above test properly:

Accurately static typing decorators in Python is an icky business. The wrapper function obfuscates type information required to statically determine the types of the parameters and the return values of the wrapped function.

Let’s write a decorator that registers the decorated functions in a global dictionary during function definition time. Here’s how I used to annotate it:

# src.py
# Import 'Callable' from 'typing' module in < Py3.9.
from collections.abc import Callable
from functools import wraps
from typing import Any, TypeVar

R = TypeVar("R")

funcs = {}


def register(func: Callable[..., R]) -> Callable[..., R]:
    """Register any function at definition time in
    the 'funcs' dict."""

    # Registers the function during function defition time.
    funcs[func.__name__] = func

    @wraps(func)
    def inner(*args: Any, **kwargs: Any) -> Any:
        return func(*args, **kwargs)

    return inner


@register
def hello(name: str) -> str:
    return f"Hello {name}!"

The functools.wraps decorator makes sure that the identity and the docstring of the wrapped function don’t get gobbled up by the decorator. This is syntactically correct and if you run Mypy against the code snippet, it’ll happily tell you that everything’s alright. However, this doesn’t exactly do anything. If you call the hello function with the wrong type of parameter, Mypy won’t be able to detect the mistake statically. Notice this:

How come I didn’t know about the python -m pydoc command before today!

It lets you inspect the docstrings of any modules, classes, functions, or methods in Python.

I’m running the commands from a Python 3.10 virtual environment but it’ll work on any Python version. Let’s print out the docstrings of the functools.lru_cache function. Run:

python -m pydoc functools.lru_cache

This will print the following on the console:

Help on function lru_cache in functools:

functools.lru_cache = lru_cache(maxsize=128, typed=False)
    Least-recently-used cache decorator.

    If *maxsize* is set to None, the LRU features are disabled and
    the cache can grow without bound.

    If *typed* is True, arguments of different types will be cached
    separately. For example, f(3.0) and f(3) will be treated as
    distinct calls with distinct results.

    Arguments to the cached function must be hashable.

    View the cache statistics named tuple (hits, misses, maxsize,
    currsize) with f.cache_info().  Clear the cache and statistics
    with f.cache_clear(). Access the underlying function with
    f.__wrapped__.

Works for third party tools as well: