Skip to main content

Comprehension debt: code that passes and nobody can explain

A Meta engineer says he ships code he doesn't fully understand. Tests pass, nobody can explain it. What comprehension debt is and how a CTO can measure it.

Comprehension debt: code that passes and nobody can explain

Ricardo Argüello

Ricardo Argüello
Ricardo Argüello

CEO & Founder

Software Development 5 min read

Ali Hamie is an engineer at Meta. On October 17 he wrote on LinkedIn that he has never felt this uncomfortable doing his job. These days, he said, he writes code and “it all feels like a black box.”

He didn’t say the code was bad. His point was about speed. If he spends his time checking each line, it “is going to take forever.” So he understands his systems at a high level, not in every detail, and he put it this way: “I’m used to understanding what I’m building.”

Addy Osmani calls this comprehension debt, the gap between how much code exists in a system and how much of it anyone actually understands. It doesn’t show up as a slow build or a fragile module. Tests pass, velocity looks great, and the bill arrives during an outage when someone has to debug a system nobody knows.

A disclosure: Osmani works at Anthropic, on Claude Code, which I use, and Anthropic published the study I cite below.

If you run an engineering team, this is the debt you aren’t tracking.

How much code goes in unread

I went to the primary sources. Sonar’s survey of more than 1,100 developers, released in January, found that 42% of committed code is AI-generated or AI-assisted, and respondents expect 65% by 2027. Of those developers, 96% don’t fully trust that code to be functionally correct. Only 48% say they always check it before committing.

Sonar ties this to what AWS CTO Werner Vogels called “verification debt.” Close cousin, different thing. Verification debt asks whether anyone reviewed the code. Comprehension debt asks whether anyone can tell you how it works. You can review a line and still not understand it.

On Meta itself, People Matters summarized a March Business Insider report (I did not read the original). It says the organization covering Facebook, WhatsApp and Messenger set a goal for 65% of its engineers to write more than 75% of their code with AI in the first half of 2026. Hamie named no team and no target. That is the backdrop, not his complaint.

What Anthropic’s study measures, and what it doesn’t

The number that circulates most comes from an Anthropic experiment. They recruited 52 engineers, mostly junior, each with over a year of weekly Python, and had them build with a library none of them knew, Trio. Then a quiz. The AI-assisted group averaged 50%. The hand-coding group averaged 67%. The widest gap was on debugging questions. Finishing was about two minutes faster with AI, and that difference was not statistically significant.

The limits matter. This measured how much a person learned about a new tool. It did not measure how well someone understands a production system they have maintained for years, and it says nothing about senior engineers. It is early evidence pointing one way. It is not a reading of your team.

What I find useful is the pattern of use. People who asked follow-up questions, requested explanations or posed conceptual questions while coding kept more. The heavy-reliance patterns, whether generating code or debugging with the AI, averaged under 40%.

What I would do

I have been programming for 36 years, since 1990, and I use Claude Code every day. What follows is opinion, not measurement.

The diff is the cheapest moment to understand a change. Afterward, understanding it costs an incident. So I wouldn’t bet on reading more lines, which fewer than half of developers say they always do. I would bet on deciding which code needs a human who understands it, and who that human is.

Four things a CTO can start this week.

First, one name per change. Each AI-written change has a person who answers for it and can explain it on request. State of AI reports that some teams already work this way.

Second, review matched to risk. An internal script can ship on tests and a quick look. A payments service needs a reviewer who can walk through every branch.

Third, make the model explain itself before the merge. Ask what it wrote and what it considered and rejected. It takes longer than accepting the diff, and it is the kind of use the Anthropic study links to keeping comprehension.

Fourth, the explain test. Once a month, pick a module an AI tool touched. Ask the person who approved it to explain it out loud without opening the tool. Count how many can. That number is the closest thing to a comprehension-debt indicator I know of, and State of AI says it has not seen any company publish one.

There is a serious counterargument. Nobody at Meta reads the compiler or every dependency, and software has always run on layers somebody else wrote. State of AI answers it well: agent-written code lands in your repository, under your name, built for your system. You are its author of record whether you can explain it or not. A compiler behaves the same way every time. This doesn’t.

On the codebase side, I wrote about why code is not cheap. On what happens to review as volume climbs, see code review for humans when 41% of the code is AI.

If you want to know how much of your code your team can explain, write to me and we’ll run the explain test on one of your modules. You pick the module.

Frequently Asked Questions

Comprehension debt is the gap between how much code a team has and how much of it anyone understands. Addy Osmani named it. Unlike technical debt, it doesn't show up as slow builds. Tests pass and dashboards look healthy until an outage forces someone to debug a system nobody knows.

There is no standard metric. A practical one is to pick a module an AI tool changed last month and ask the person who approved the change to explain it without opening the tool. Count how many can. That count is a working measure of comprehension debt.

Anthropic compared 52 mostly junior engineers learning an unfamiliar Python library. The group with an AI assistant scored 50% on the follow-up quiz and the hand-coding group scored 67%, with the largest gap on debugging. Engineers who asked the AI questions retained more than those who delegated the code.

Name one person for each AI-written change who must be able to explain it, match review depth to the risk of the code, and ask the model to explain what it wrote and what it rejected. An internal script and a payments service should not get the same review.

comprehension debt Ali Hamie Meta Addy Osmani Sonar Claude Code code review

Related Articles

Claude Code Review is GA. Who signs off on the merge?
Software Development
· 5 min read

Claude Code Review is GA. Who signs off on the merge?

Anthropic opened Code Review to Team and Enterprise. It never approves or blocks a PR. What you still decide: who signs off, what it costs, what stops a merge.

Claude Code Review code review AI governance
The hidden cost of AI coding is review, not writing
Software Development
· 6 min read

The hidden cost of AI coding is review, not writing

Business Insider calls it workplace paralysis. The hidden cost of AI coding isn't the AI. It's that code now ships faster than any human can review or own it.

AI coding code review cognitive debt
Your Code Review Was Built for Humans. 41% of Code Isn't
Software Development
· 8 min read

Your Code Review Was Built for Humans. 41% of Code Isn't

AI code review when 41% of code is AI-generated and carries 1.7x more defects: the review steps, CI/CD quality gates and code quality checks your team needs.

code quality code review AI-generated code
What Your AI Won't Ask (and Your Startup Will Pay)
Software Development
· 6 min read

What Your AI Won't Ask (and Your Startup Will Pay)

A founder lost $87,500 because his AI generated working code without questioning security. AI tools answer what you ask, not what's missing.

vibe coding software security startups
Muse Spark Open: What Are You Paying a Closed Vendor For?
Business Strategy
· 5 min read

Muse Spark Open: What Are You Paying a Closed Vendor For?

Meta open-sourced Muse Spark Open and seven hosts already sell it. If an open model covers most of your work, a closed API is billing you for the hard tail.

Muse Spark Open Meta open-weight models
The US Needs 500,000 Electricians to Build AI
Business Strategy
· 5 min read

The US Needs 500,000 Electricians to Build AI

Meta says the US needs 500,000 electricians for AI infrastructure. Ford, Google, BlackRock, and Carhartt already put $450 million behind the problem.

AI infrastructure skilled trades shortage data centers