Shell

I’m not really a fan of shims—code that automatically performs actions as a side effect or intercepts commands when you use the shell or when a prompt runs. That’s why, other than the occasional dabbling, I’ve mostly stayed away from tools like asdf or pyenv and instead stick to apt or brew for managing my binary installs, depending on the OS. Recently, though, I’ve started seeing many people I admire extolling direnv: ...

While going through a script at work today, I came across Bash’s nameref feature. It uses declare -n ref="$1" to set up a variable that allows you to reference another variable by name—kind of like pass-by-reference in C. I’m pretty sure I’ve seen it before, but I probably just skimmed over it. As I dug into the man page1, I realized there’s a gap in my understanding of how variable references actually work in Bash—probably because I never gave it proper attention and just got by cobbling together scripts. ...

I always struggle with the syntax for redirecting multiple streams to another command or a file. LLMs do help, but beyond the most obvious cases, it takes a few prompts to get the syntax right. When I know exactly what I’m after, scanning a quick post is much faster than wrestling with a non-deterministic kraken. So, here’s a list of the redirection and piping syntax I use the most, with real examples. ...

I learned this neat Bash trick today where you can make a raw HTTP request using the /dev/tcp file descriptor without using tools like curl or wget. This came in handy while writing a health check script that needed to make a TCP request to a service. The following script opens a TCP connection and makes a simple GET request to example.com: #!/bin/bash # Open a TCP connection to example.com on port 80 and assign file descriptor 3 # The exec command keeps /dev/fd/3 open throughout the lifetime of the script # 3<> enables bidirectional read-write exec 3<>/dev/tcp/example.com/80 # Send the HTTP GET request to the server # >& redirects stdout to /dev/fd/3 echo -e "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n" >&3 # Read and print the server's response # <& redirects the output of /dev/fd/3 to cat cat <&3 # Close the file descriptor, terminating the TCP connection exec 3>&- Running this will print the response from the site to your console. ...

TIL about the install command on *nix systems. A quick GitHub search for the term brought up a ton of matches1. I’m surprised I just found out about it now. Often, in shell scripts I need to: Create a directory hierarchy Copy a config or binary file to the new directory Set permissions on the file It usually looks like this: # Create the directory hierarchy. The -p flag creates the parent directories # if they don't exist mkdir -p ~/.config/app # Copy the current config to the newly created directory. Here, conf already # exists in the current folder cp conf ~/.config/app/conf # Set the file permission chmod 755 ~/.config/app/conf Turns out, the install command in GNU coreutils2 can do all that in one line: ...

I was working on the deployment pipeline for a service that launches an app in a dedicated VM using GitHub Actions. In the last step of the workflow, the CI SSHs into the VM and runs several commands using a here document1 in bash. The simplified version looks like this: # SSH into the remote machine and run a bunch of commands to deploy the service ssh $SSH_USER@$SSH_HOST <<EOF # Go to the work directory cd $WORK_DIR # Make a git pull git pull # Export environment variables required for the service to run export AUTH_TOKEN=$APP_AUTH_TOKEN # Start the service docker compose up -d --build EOF The fully working version can be found on GitHub2. ...

I like writing custom scripts to automate stuff or fix repetitive headaches. Most of them are shell scripts, and a few of them are written in Python. Over the years, I’ve accumulated quite a few of them. I use Git and GNU stow1 to manage them across different machines, and the workflow2 is quite effective. However, as the list of scripts grows larger, invoking them becomes a pain because the tab completion results get cluttered with other system commands. Plus, often I even forget the initials of a script’s name and stare at my terminal while the blinking cursor facepalms at my stupidity. ...

I’m one of those people who will sit in front of a computer for hours, fiddling with algorithms or debugging performance issues, yet won’t spend 10 minutes to improve their workflows. While I usually get away with this, every now and then, my inertia slithers back to bite me. The latest episode was me realizing how tedious it is to move config files across multiple devices when I was configuring a new MacBook Air and Mac Mini at the same time. ...

This morning, while browsing Hacker News, I came across a neat trick1 that allows you to share textual data by leveraging DNS TXT records. It can be useful for sharing a small amount of data in environments that restrict IP but allow DNS queries, or to bypass censorship. To test this out, I opened my domain registrar’s panel and created a new TXT type DNS entry with a base64 encoded message containing the poem A Poison Tree by William Blake. The message can now be queried and decoded with the following shell command: ...

I was watching this amazing lightning talk1 by Karla Burnett and wanted to understand how traceroute works in Unix. Traceroute is a tool that shows the route of a network packet from your computer to another computer on the internet. It also tells you how long it takes for the packet to reach each stop along the way. It’s useful when you want to know more about how your computer connects to other computers on the internet. For example, if you want to visit a website, your computer sends a request to the website’s server, which is another computer that hosts the website. But the request doesn’t go directly from your computer to the server. It has to pass through several other devices, such as routers, that help direct the traffic on the internet. These devices are called hops. Traceroute shows you the list of hops that your request goes through, and how long it takes for each hop to respond. This can help you troubleshoot network problems, such as slow connections or unreachable websites. ...

This weekend, I was working on a fun project that required a fixed-time job scheduler to run a curl command at a future timestamp. I was aiming to find the simplest solution that could just get the job done. I’ve also been exploring Google Bard1 recently and wanted to see how it stacks up against other LLM tools like ChatGPT, BingChat, or Anthropic’s Claude in terms of resolving programming queries. ...

One of my favorite pastimes these days is to set BingChat to creative mode, ask it to teach me a trick about topic X, and then write a short blog post about it to reinforce my understanding. Some of the things it comes up with are absolutely delightful. In the spirit of that, I asked it to teach me a Shell trick that I can use to mimic maps or dictionaries in a shell environment. I didn’t even know what I was expecting. ...

I needed to compare two large directories with thousands of similarly named PDF files and find the differing filenames between them. In the first pass, this is what I did: Listed out the content of the first directory and saved it in a file: ls dir1 > dir1.txt Did the same for the second directory: ls dir2 > dir2.txt Compared the difference between the two outputs: diff dir1.txt dir2.txt This returned the name of the differing files likes this: ...

Whenever I need to whip up a quick command line tool, my go-to is usually Python. Python’s CLI solutions tend to be more robust than their Shell counterparts. However, dealing with its portability can sometimes be a hassle, especially when all you want is to distribute a simple script. That’s why while toying around with argparse to create a dynamic menu, I decided to ask ChatGPT if there’s a way to achieve the same using native shell scripting. Delightfully, it introduced me to the dead-simple select command that I probably should’ve known about years ago. But I guess better late than never! Here’s what I was trying to accomplish: ...

When writing shell scripts, I’d often resort to using hardcoded ANSI escape codes1 to format text, such as: #!/usr/bin/env bash BOLD="\033[1m" UNBOLD="\033[22m" FG_RED="\033[31m" BG_YELLOW="\033[43m" BG_BLUE="\033[44m" RESET="\033[0m" # Print a message in bold red text on a yellow background. echo -e "${BOLD}${FG_RED}${BG_YELLOW}This is a warning message${RESET}" # Print a message in white text on a blue background. echo -e "${BG_BLUE}This is a debug message${RESET}" This shell snippet above shows how to add text formatting and color to shell script output via ANSI escape codes. It defines a few variables that contain different escape codes for bold, unbold, foreground, and background colors. Then, we echo two log messages with different colors and formatting options. ...

I’ve always had a vague idea about what Unix domain sockets are from my experience working with Docker for the past couple of years. However, lately, I’m spending more time in embedded edge environments and had to explore Unix domain sockets in a bit more detail. This is a rough documentation of what I’ve explored to gain some insights. The dry definition Unix domain sockets (UDS) are similar to TCP sockets in a way that they allow two processes to communicate with each other, but there are some core differences. While TCP sockets are used for communication over a network, Unix domain sockets are used for communication between processes running on the same computer. ...

The colon : command is a shell utility that represents a truthy value. It can be thought of as an alias for the built-in true command. You can test it by opening a shell script and typing a colon on the command line, like this: : If you then inspect the exit code by typing $? on the command line, you’ll see a 0 there, which is exactly what you’d see if you had used the true command. ...

My grug1 brain can never remember the correct semantics of quoting commands and variables in a UNIX shell environment. Every time I work with a shell script or run some commands in a Docker compose file, I’ve to look up how to quote things properly to stop my ivory tower from crashing down. So, I thought I’d list out some of the most common rules that I usually look up all the time. ...

TIL that returning a value from a function in bash doesn’t do what I thought it does. Whenever you call a function that’s returning some value, instead of giving you the value, Bash sets the return value of the callee as the status code of the calling command. Consider this example: #!/usr/bin/bash # script.sh return_42() { return 42 } # Call the function and set the return value to a variable. value=$return_42 # Print the return value. echo $value I was expecting this to print out 42 but instead it doesn’t print anything to the console. Turns out, a shell function doesn’t return the value when it encounters the return keyword. Rather, it stops the execution of the function and sets the status code of the last command in the function as the value that the function returns. ...

I run git log --oneline to list out the commit logs all the time. It prints out a compact view of the git history. Running the command in this repo gives me this: d9fad76 Publish blog on safer operator.itemgetter, closes #130 0570997 Merge pull request #129 from rednafi/dependabot/... 6967f73 Bump actions/setup-python from 3 to 4 48c8634 Merge pull request #128 from rednafi/dependabot/pip/mypy-0.961 5b7a7b0 Bump mypy from 0.960 to 0.961 However, there are times when I need to list out the commit logs that only represent the changes made to a particular file. Here’s the command that does exactly that. ...