The Hidden Technical Debt in LLM-Generated Code

As a rule, I generally avoid speaking on subjects I know nothing, or not enough, about. In the last few years working in AI, I have felt that way constantly. Like it’s all moving too fast for any one human to speak about with real authority, as the technical developments abound at a rate that makes my head spin daily. I spend my entire morning commute trying to catch up on the latest AI release, where the bubble has pushed us to now, and how the smartest or most innovative of us are using these tools.

I spend my nights trying to drown out the noise of people who don’t know enough to speak on these subjects as I doomscroll through silly little videos of literally anything else.

With that said, I gave a talk this summer on deterministic testing in agentic systems. Really exciting stuff, I know. And while the talk went well, my biggest takeaways had nothing to do with testing and everything to do with LLM competency. I was, I regret to say, surprised to find the general level of understanding of agentic and LLM-driven programming lacking, given that I was speaking to a room full of engineers.

These engineers are certainly good at their jobs and strive to use the latest tools, but perhaps they don’t work at an AI startup where they can really stretch their wings.

It was inspirational for me, as I generally suffer from the “I know it now, so everyone must know it” disease, where I am convinced that my level of understanding on a topic is shared across my entire peer group. Call it imposter syndrome if you must, but I digress.

So that brings us to today, when I merged a particularly large PR I wasn’t particularly proud of, and decided to publicly reflect on why.

I am mocked at work for being especially militant in code reviews, always have been. But in recent months, something has changed rather dramatically in our process as a team. The quality of the code LLMs are producing, combined with the speed of delivery we need at this stage of our company, has changed the value proposition of ‘vibe coding’ features in a positive way. I've been the biggest deterrent to this. Shipping code I don’t fully understand is my worst nightmare, as I have enough years under my belt to be thoroughly stressed about what can go wrong in production. But the decision about which trade-offs to make is constant and shifting.

With all this in mind, I want to talk about this most recent PR. It was a feature that allows us to support trial and converted accounts, which, when I originally scoped it, seemed quite trivial. I wrote a product roadmap document and went into what I thought was sufficient detail, as I usually do. However, as the feature scope crept and the true tendrils of the change revealed themselves, I realized how poorly I had scoped the implementation. What was supposed to be a one-day implementation became 17 days.

So what went wrong?

To my knowledge, I have not gotten dumber (although with an increasing reliance on LLMs, I wouldn’t rule it out). But I realized that I am confronting the true threat that no one is talking about with regard to LLM code debt, and that I had stumbled upon an important choice that engineering organizations are making and will continue to make.

When we talk about code debt (the software engineering term for the future time it will take to fix bad coding decisions in the codebase) with regard to LLMs, we aren’t talking only about code quality. They sometimes produce code that is dramatically worse or dramatically better than a human would, largely depending on the human in question. However, the issue usually isn’t the first vibe-coded feature; that works. The risk I ran into was much scarier.

Over the past 6 months, so many complexities have been added to our system that I didn’t fully understand. That’s not because I haven’t been keeping up; I review nearly every PR my team pushes into the codebase. However, I hadn’t really internalized even the features I authored. I read all the code, but any software engineer will tell you that’s a far cry from writing the code myself and obsessing over every technical and architectural detail. The depth of knowledge is dramatically different. Instead of stressing over architecture decisions before we make them, we are now reading the LLM's decisions retroactively and deciding whether they are passable.

As a result of this shift, my ability to judge the system's complexity was poor. I no longer know my own codebase the way I used to, and that is its own new kind of tech debt. Upon this realization, I had a shuddering flashback to the very worst engineering managers I have worked with in my career: those who purported to have an opinion on my work but lacked deep knowledge of the codebase and the real tradeoffs, and so wasted my time by requiring me to get them back up to speed. This implementation felt like that, but I was all alone, just me and an Opus instance against my impending fear and a deadline I had blown through.

So what does this mean for the future of SWE, or our company’s approach to it? I’m not sure.

I know that we can’t afford to slow down. I know we are smart enough to keep diligently reviewing our code for real, dangerous errors. But I also know this disconnection will only grow.

Does it matter?

We debate the probability that these poor architecture decisions will become real issues in the future. What if the future codebase is 100% supported by continually improving LLMs? Won’t they be able to find and address our shortcomings today? Should I be less afraid of 50-file-change PRs? Or will the issue grow until we come to a screeching halt and have to allocate 6 months to cleaning up the mess we’ve made? That's the million-dollar question, isn’t it?

IMHO, it will likely fall somewhere in the middle, and the best course of action is to use the momentum to build systems that handle the bumpy road with grace. But as with all new technologies that materially shift our lives and ways of working, only time will tell.

‍