Typing

Although I’ve been using Python 3.12 in production for nearly a year, one neat feature in the typing module that escaped me was the @override decorator. Proposed in PEP 698, it’s been hanging out in typing_extensions for a while. This is one of those small features you either don’t care about or get totally psyched over. I’m definitely in the latter camp. In languages like C#, Java, and Kotlin, explicit overriding is required. For instance, in Java, you use @Override to make it clear you’re overriding a method in a sub class. If you mess up the method name or if the method doesn’t exist in the superclass, the compiler throws an error. Now, with Python’s @override decorator, we get similar benefits - though only if you’re using a static type checker. ...

The handful of times I’ve reached for typing.TypeGuard in Python, I’ve always been confused by its behavior and ended up ditching it with a # type: ignore comment. For the uninitiated, TypeGuard allows you to apply custom type narrowing. For example, let’s say you have a function named pretty_print that accepts a few different types and prints them differently onto the console: from typing import assert_never def pretty_print(val: int | float | str) -> None: if isinstance(val, int): # assert_type(val, int) print(f"Integer: {val}") elif isinstance(val, float): # assert_type(val, float) print(f"Float: {val}") elif isinstance(val, str): # assert_type(val, str) print(f"String: {val}") else: assert_never(val) If you run it through mypy, in each branch, the type checker automatically narrows the type and knows exactly what the type of val is. You can test the narrowed type in each branch with the typing.assert_type function. ...

While I tend to avoid *args and **kwargs in my function signatures, it’s not always possible to do so without hurting API ergonomics. Especially when you need to write functions that call other helper functions with the same signature. Typing *args and **kwargs has always been a pain since you couldn’t annotate them precisely before. For example, if all the positional and keyword arguments of a function had the same type, you could do this: ...

You can use @dataclass(frozen=True) to make instances of a data class immutable during runtime. However, there’s a small caveat - instantiating a frozen data class is slightly slower than a non-frozen one. This is because, when you enable frozen=True, Python has to generate __setattr__ and __delattr__ methods during class definition time and invoke them for each instantiation. Below is a quick benchmark comparing the instantiation times of a mutable dataclass and a frozen one (in Python 3.12): ...

Nested conditionals suck. They’re hard to write and even harder to read. I’ve rarely regretted the time I’ve spent optimizing for the flattest conditional structure in my code. The following piece mimics the actions of a traffic signal: // src.ts enum Signal { YELLOW = "Yellow", RED = "Red", GREEN = "Green", } function processSignal(signal: Signal) :void { if (signal === Signal.YELLOW) { console.log("Slow down!"); } else { if (signal === Signal.RED) { console.log("Stop!"); } else { if (signal === Signal.GREEN) { console.log("Go!"); } } } } // Log processSignal(Signal.YELLOW) // prints 'Slow down!' processSignal(Signal.RED) // prints 'Stop!' The snippet above suffers from two major issues: ...

While working with microservices in Python, a common pattern that I see is - the usage of dynamically filled dictionaries as payloads of REST APIs or message queues. To understand what I mean by this, consider the following example: # src.py from __future__ import annotations import json from typing import Any import redis # Do a pip install. def get_payload() -> dict[str, Any]: """Get the 'zoo' payload containing animal names and attributes.""" payload = {"name": "awesome_zoo", "animals": []} names = ("wolf", "snake", "ostrich") attributes = ( {"family": "Canidae", "genus": "Canis", "is_mammal": True}, {"family": "Viperidae", "genus": "Boas", "is_mammal": False}, ) for name, attr in zip(names, attributes): payload["animals"].append( # type: ignore {"name": name, "attribute": attr}, ) return payload def save_to_cache(payload: dict[str, Any]) -> None: # You'll need to spin up a Redis db before instantiating # a connection here. r = redis.Redis() print("Saving to cache...") r.set(f"zoo:{payload['name']}", json.dumps(payload)) if __name__ == "__main__": payload = get_payload() save_to_cache(payload) Here, the get_payload function constructs a payload that gets stored in a Redis DB in the save_to_cache function. The get_payload function returns a dict that denotes a contrived payload containing the data of an imaginary zoo. To execute the above snippet, you’ll need to spin up a Redis database first. You can use Docker to do so. Install and configure Docker on your system and run: ...

PEP-673 introduces the Self type and it’s coming to Python 3.11. However, you can already use that now via the typing_extensions module. The Self type makes annotating methods that return the instances of the corresponding classes trivial. Before this, you’d have to do some mental gymnastics to statically type situations as follows: # src.py from __future__ import annotations from typing import Any class Animal: def __init__(self, name: str, says: str) -> None: self.name = name self.says = says @classmethod def from_description(cls, description: str = "|") -> Animal: descr = description.split("|") return cls(descr[0], descr[1]) class Dog(Animal): def __init__(self, *args: Any, **kwargs: Any) -> None: super().__init__(*args, **kwargs) @property def legs(self) -> int: return 4 if __name__ == "__main__": dog = Dog.from_description("Matt | woof") print(dog.legs) # Mypy complains here! The class Animal has a from_description class method that acts as an additional constructor. It takes a description string, and then builds and returns an instance of the same class. The return type of the method is annotated as Animal here. However, doing this makes the child class Dog conflate its identity with the Animal class. If you execute the snippet, it won’t raise any runtime error. Also, Mypy will complain about the type: ...

Static type checkers like Mypy follow your code flow and statically try to figure out the types of the variables without you having to explicitly annotate inline expressions. For example: # src.py from __future__ import annotations def check(x: int | float) -> str: if not isinstance(x, int): reveal_type(x) # Type is now 'float'. else: reveal_type(x) # Type is now 'int'. return str(x) The reveal_type function is provided by Mypy and you don’t need to import this. But remember to remove the function before executing the snippet. Otherwise, Python will raise a runtime error as the function is only understood by Mypy. If you run Mypy against this snippet, it’ll print the following lines: ...

Technically, the type of None in Python is NoneType. However, you’ll rarely see types.NoneType being used in the wild as the community has pretty much adopted None to denote the type of the None singleton. This usage is also documented in PEP-484. Whenever a callable doesn’t return anything, you usually annotate it as follows: # src.py from __future__ import annotations def abyss() -> None: return But sometimes a callable raises an exception and never gets the chance to return anything. Consider this example: ...

While grokking Black formatter’s codebase, I came across this Rust-influenced error handling model that offers an interesting way of handling exceptions in Python. Exception handling in Python usually follows the EAFP paradigm where it’s easier to ask for forgiveness than permission. However, Rust has this recoverable error handling workflow that leverages generic Enums. I wanted to explore how Black emulates that in Python. This is how it works: # src.py from __future__ import annotations from typing import Generic, TypeVar, Union T = TypeVar("T") E = TypeVar("E", bound=Exception) class Ok(Generic[T]): def __init__(self, value: T) -> None: self._value = value def ok(self) -> T: return self._value class Err(Generic[E]): def __init__(self, e: E) -> None: self._e = e def err(self) -> E: return self._e Result = Union[Ok[T], Err[E]] In the above snippet, two generic types Ok and Err represent the return type and the error types of a callable respectively. These two generics were then combined into one Result generic type. You’d use the Result generic to handle exceptions as follows: ...

I’ve always had a hard time explaining variance of generic types while working with type annotations in Python. This is an attempt to distill the things I’ve picked up on type variance while going through PEP-483. A pinch of type theory A generic type is a class or interface that is parameterized over types. Variance refers to how subtyping between the generic types relates to subtyping between their parameters' types. ...

Accurately static typing decorators in Python is an icky business. The wrapper function obfuscates type information required to statically determine the types of the parameters and the return values of the wrapped function. Let’s write a decorator that registers the decorated functions in a global dictionary during function definition time. Here’s how I used to annotate it: # src.py # Import 'Callable' from 'typing' module in < Py3.9. from collections.abc import Callable from functools import wraps from typing import Any, TypeVar R = TypeVar("R") funcs = {} def register(func: Callable[..., R]) -> Callable[..., R]: """Register any function at definition time in the 'funcs' dict.""" # Registers the function during function defition time. funcs[func.__name__] = func @wraps(func) def inner(*args: Any, **kwargs: Any) -> Any: return func(*args, **kwargs) return inner @register def hello(name: str) -> str: return f"Hello {name}!" The functools.wraps decorator makes sure that the identity and the docstring of the wrapped function don’t get gobbled up by the decorator. This is syntactically correct and if you run Mypy against the code snippet, it’ll happily tell you that everything’s alright. However, this doesn’t exactly do anything. If you call the hello function with the wrong type of parameter, Mypy won’t be able to detect the mistake statically. Notice this: ...

If you want to define a variable that can accept values of multiple possible types, using typing.Union is one way of doing that: from typing import Union U = Union[int, str] However, there’s another way you can express a similar concept via constrained TypeVar. You’d do so as follows: from typing import TypeVar T = TypeVar("T", int, str) So, what’s the difference between these two and when to use which? The primary difference is: T’s type needs to be consistent across multiple uses within a given scope, while U’s doesn’t. ...

I love using Go’s interface feature to declaratively define my public API structure. Consider this example: package main import ( "fmt" ) // Declare the interface. type Geometry interface { area() float64 perim() float64 } // Struct that represents a rectangle. type rect struct { width, height float64 } // Method to calculate the area of a rectangle instance. func (r *rect) area() float64 { return r.width * r.height } // Method to calculate the perimeter of a rectange instance. func (r *rect) perim() float64 { return 2 * (r.width + r.height) } // Notice that we're calling the methods on the interface, // not on the instance of the Rectangle struct directly. func measure(g Geometry) { fmt.Println(g) fmt.Println(g.area()) fmt.Println(g.perim()) } func main() { r := &rect{width: 3, height: 4} measure(r) } You can play around with the example on Go Playground. Running it will print: ...