Database

I was working on a project where I needed to poll multiple data sources and consume the incoming data points in a single thread. In this particular case, the two data streams were coming from two different Redis lists. The correct way to consume them would be to write two separate consumers and spin them up as different processes. However, in this scenario, I needed a simple way to poll and consume data from one data source, wait for a bit, then poll and consume from another data source, and keep doing this indefinitely....

Django has a Model.objects.bulk_update method that allows you to update multiple objects in a single pass. While this method is a great way to speed up the update process, oftentimes it’s not fast enough. Recently, at my workplace, I found myself writing a script to update half a million user records and it was taking quite a bit of time to mutate them even after leveraging bulk update. So I wanted to see if I could use multiprocessing with ....

While going through the documentation of Python’s sqlite31 module, I noticed that it’s quite API-driven, where different parts of the module are explained in a prescriptive manner. I, however, learn better from examples, recipes, and narratives. Although a few good recipes already exist in the docs, I thought I’d also enlist some of the examples I tried out while grokking them. Executing individual statements To execute individual statements, you’ll need to use the cursor_obj....

Python has a random.choice routine in the standard library that allows you to pick a random value from an iterable. It works like this: # src.py import random # The seed ensures that you'll get the same random choice # every time you run the script. random.seed(90) # This builds a list: ["choice_0", "choice_1", ..., "choice_9"] lst = [f"choice_{i}" for i in range(10)] print(random.choice(lst)) print(random.choice(lst)) This will print: choice_3 choice_1 I was looking for a way to quickly hydrate a table with random data in an SQLite database....