Shell

I came across a weird shell syntax today - dynamic shell variables. It lets you dynamically construct and access variable names in Bash scripts, which I haven’t encountered in any of the mainstream languages I juggle for work. In an actual programming language, you’d usually use a hashmap to achieve the same effect, but directly templating variable names is a quirky shell feature that sometimes comes in handy. A primer Dynamic shell variables allow shell scripts to define and access variables based on runtime conditions. Variable indirection (${!var} syntax) lets you reference the value of a variable through another variable. This can be useful for managing environment-specific configurations and function dispatch mechanisms. ...

I’ve been having a ton of fun fiddling with Tailscale over the past few days. While setting it up on a server, I came across this ufw firewall script that configures the firewall on Linux to ensure direct communication across different nodes in my tailnet. It has the following block of code that I found interesting (added comments for clarity): #!/usr/bin/env bash # Define PID file path using script's name for uniqueness PIDFILE="/tmp/$(basename "${BASH_SOURCE[0]%.*}.pid")" # Open file descriptor 200 for the PID file exec 200>"${PIDFILE}" # Try to acquire a non-blocking lock; exit if the script is already running flock -n 200 \ || { echo "${BASH_SOURCE[0]} already running. Aborting..."; exit 1; } # Store the current process ID (PID) in the lock file for reference PID=$$ echo "${PID}" 1>&200 # Do work (in the original script, real work happens here) sleep 999 Here, flock is a Linux command that ensures only one instance of the script runs at a time by locking a specified file (e.g., PIDFILE) through a file descriptor (e.g., 200). If another process already holds the lock, the script either waits or exits immediately. Above, it bails with an error message and exit code 1. ...

Setting up SSH access to a new VM usually follows the same routine: generate a key pair, copy it to the VM, tweak some configs, confirm the host’s identity, and maybe set up an agent to avoid typing passphrases all day. Tools like cloud-init and Ansible handle most of the setup for me now, so I rarely think about it. But I realized I don’t fully understand how all the parts work together. ...

I’m not a big fan of shims - code that messes with commands in the shell or prompt. That’s why, aside from occasional dabbling, I tend to eschew tools like asdf or pyenv and just use apt or brew for installs, depending on the OS. Then recently, I saw Hynek extolling direnv: If you’re old-school like me, my .envrc looks like this: uv sync --frozen source .venv/bin/activate The sync ensures there’s always a .venv, so no memory-baking required. ...

While going through a script at work today, I came across Bash’s nameref feature. It uses declare -n ref="$1" to set up a variable that allows you to reference another variable by name - kind of like pass-by-reference in C. I’m pretty sure I’ve seen it before, but I probably just skimmed over it. As I dug into the man pages, I realized there’s a gap in my understanding of how variable references actually work in Bash - probably because I never gave it proper attention and just got by cobbling together scripts. ...

I always struggle with the syntax for redirecting multiple streams to another command or a file. LLMs do help, but beyond the most obvious cases, it takes a few prompts to get the syntax right. When I know exactly what I’m after, scanning a quick post is much faster than wrestling with a non-deterministic kraken. So, here’s a list of the redirection and piping syntax I use the most, with real examples. ...

I learned this neat Bash trick today where you can make a raw HTTP request using the /dev/tcp file descriptor without using tools like curl or wget. This came in handy while writing a health check script that needed to make a TCP request to a service. The following script opens a TCP connection and makes a simple GET request to example.com: #!/bin/bash # Open TCP connection to example.com:80 and assign file descriptor 3 # exec keeps /dev/fd/3 open; 3<> enables bidirectional read-write exec 3<>/dev/tcp/example.com/80 # Send the HTTP GET request to the server (>& redirects to /dev/fd/3) echo -e \ "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n" >&3 # Read and print the server's response # <& redirects the output of /dev/fd/3 to cat cat <&3 # Close the file descriptor, terminating the TCP connection exec 3>&- Running this will print the response from the site to your console. ...

TIL about the install command on *nix systems. A quick GitHub search for the term brought up a ton of matches. I’m surprised I just found out about it now. Often, in shell scripts I need to: Create a directory hierarchy Copy a config or binary file to the new directory Set permissions on the file It usually looks like this: # Create directory hierarchy (-p creates parent directories) mkdir -p ~/.config/app # Copy current config to the newly created directory cp conf ~/.config/app/conf # Set the file permission chmod 755 ~/.config/app/conf Turns out, the install command in GNU coreutils can do all that in one line: ...

I was working on the deployment pipeline for a service that launches an app in a dedicated VM using GitHub Actions. In the last step of the workflow, the CI SSHs into the VM and runs several commands using a here document in bash. The simplified version looks like this: # SSH into the remote machine and run commands to deploy the service ssh $SSH_USER@$SSH_HOST <<EOF # Go to the work directory cd $WORK_DIR # Make a git pull git pull # Export environment variables required for the service to run export AUTH_TOKEN=$APP_AUTH_TOKEN # Start the service docker compose up -d --build EOF The fully working version can be found in the serve-init repo with here-doc. ...

People tend to get pretty passionate about Git workflows on different online forums. Some like to rebase, while others prefer to keep the disorganized records. Some dislike the extra merge commit, while others love to preserve all the historical artifacts. There’s merit to both sides of the discussion. That being said, I kind of like rebasing because I’m a messy committer who: Usually doesn’t care for keeping atomic commits. Creates a lot of short commits with messages like “fix” or “wip”. Likes to clean up the untidy commits before sending the branch for peer review. Prefers a linear history over a forked one so that git log --oneline --graph tells a nice story. Git rebase allows me to squash my disordered commits into a neat little one, which bundles all the changes with passing tests and documentation. Sure, a similar result can be emulated using git merge --squash feat_branch or GitHub’s squash-merge feature, but to me, rebasing feels cleaner. Plus, over time, I’ve subconsciously picked up the tricks to work my way around rebase-related gotchas. ...

I like writing custom scripts to automate stuff or fix repetitive headaches. Most of them are shell scripts, and a few of them are written in Python. Over the years, I’ve accumulated quite a few of them. I use Git and GNU stow to manage them across different machines, and the dotfile workflow is quite effective. However, as the list of scripts grows larger, invoking them becomes a pain because the tab completion results get cluttered with other system commands. Plus, often I even forget the initials of a script’s name and stare at my terminal while the blinking cursor facepalms at my stupidity. ...

I’m one of those people who will sit in front of a computer for hours, fiddling with algorithms or debugging performance issues, yet won’t spend 10 minutes to improve their workflows. While I usually get away with this, every now and then, my inertia slithers back to bite me. The latest episode was me realizing how tedious it is to move config files across multiple devices when I was configuring a new MacBook Air and Mac Mini at the same time. ...

This morning, while browsing Hacker News, I came across a neat trick for sharing data via DNS TXT records. It can be useful for propagating a small amount of data in environments that restrict IP but allow DNS queries, or to bypass censorship. To test this out, I opened my domain registrar’s panel and created a new TXT type DNS entry with a base64 encoded message containing the poem A Poison Tree by William Blake. The message can now be queried and decoded with the following shell command: ...

Python offers a ton of ways like os.system or os.spawn* to create new processes and run arbitrary commands in your system. However, the documentation usually encourages you to use the subprocess module for creating and managing child processes. The subprocess module exposes a high-level run() function that provides a simple interface for running a subprocess and waiting for it to complete. It accepts the command to run as a list of strings, starts the subprocess, waits for it to finish, and then returns a CompletedProcess object with information about the result. For example: ...

I was watching Storytelling with traceroute lightning talk by Karla Burnett and wanted to understand how traceroute works in Unix. Traceroute is a tool that shows the route of a network packet from your computer to another computer on the internet. It also tells you how long it takes for the packet to reach each stop along the way. It’s useful when you want to know more about how your computer connects to other computers on the internet. For example, if you want to visit a website, your computer sends a request to the website’s server, which is another computer that hosts the website. But the request doesn’t go directly from your computer to the server. It has to pass through several other devices, such as routers, that help direct the traffic on the internet. These devices are called hops. Traceroute shows you the list of hops that your request goes through, and how long it takes for each hop to respond. This can help you troubleshoot network problems, such as slow connections or unreachable websites. ...