Ever been in a situation where you landed a software engineering job with a particular tech stack, mastered it, switched to another company with a different stack, nailed that too, and then found yourself in a third company that used the original stack? Now, you suddenly sense that your hard-earned acumen in that initial stack has not only atrophied over the years but also a portion, or all of it, has become irrelevant, making it a bit of a struggle to catch up with the latest changes.
After graduation, I switched gears from electrical to software engineering. I started out as a junior data scientist at an e-commerce startup. There, I juggled tasks like training small-scale machine learning models, analyzing tabular data, and building visualizations with the Python data stack. When COVID hit, I jumped ship to another company using a different tech stack, shifting my attention to distributed system and backend engineering.
In my current role, I’m still mostly doing backend work, just with a different set of tools than before. Throughout this journey, I’ve been fortunate enough to dart around three different continents. While hopping between positions and tech stacks has definitely widened my horizon, I’ve been grappling with the observation that as I pick up new skills, some of the older ones depreciate at a faster pace. It’s quite difficult to keep yourself sharp with tools that you don’t get to use regularly at work.
With all the hype around LLMs lately, I’m feeling drawn back to my original data science roots. To rekindle that part of my brain, I’ve started dabbling in some of my old OSS work, and to my great surprise, I’m struggling quite a bit to pick up the fundamentals and the required mathematics since I’ve been out of the game for so long. While I’ve kept in touch with some part of the open-source data world to stay relevant, apparently that wasn’t enough. On top of that, the world keeps piling on new concepts and skills that I’ll need to pick up if I ever intend to get past the interview barrier and professionally work in this arena again.
Turns out this manifestation of stochastic knowledge decay is a well-studied phenomenon. The term half-life of knowledge1 was coined in 1962 by economist Fritz Machlup2 to represent the amount of time that has to elapse before half of the knowledge or facts in a particular area is superseded or shown to be untrue. It’s named after the half-life of decay in radioactive materials. This article3 on the IEEE Spectrum dives deep into the concept and reflects upon its effect on the industry. It postulates that the half-life of engineering knowledge is shrinking faster than ever before and the only way to tackle this is through continuous learning and getting better at managing the onslaught of information.
I don’t have a prescriptive solution for this. I wrote this text to start a discussion around a feeling I previously struggled with but didn’t know how to label. So far, engaging with the OSS community on topics I find exciting, taking meticulous notes, tracking my learning progress, adopting boring technology, and writing about them have helped me stay relevant. However, this approach isn’t bulletproof and is quite susceptible to lack of motivation at the tail end of a 40-hour workweek. If you’ve experienced something similar and found a solution that worked for you to some extent, I’d love to hear about it!