Last week I wrote about DHH’s claim that programmers may be worse at AI coding than people who cannot code. The piece argued the headline version of that claim is wrong in its strong form, and that the fair version is narrower: for problems where product sense dominates, an experienced programmer who over-specifies can get worse agent results than a non-programmer who just describes the problem.
Since then, two things happened that deserve a follow-up. A preregistered study from ETH Zurich landed with actual numbers on who succeeds at vibe coding. And a post about that study by @thesupermanmx pulled over four thousand likes at the time of writing by presenting it as proof of the opposite of DHH’s argument. Both are worth reading. Neither settles the question, but together they sharpen it in a way neither does alone.
What the study actually measured
The paper, Thorgeirsson, Weidmann and Su at CHI 2026, is a preregistered controlled study of vibe coding proficiency. One hundred students from ETH Zurich and the University of Zurich each took a computer science achievement test, a general reasoning test, and a graded essay. Then they completed three timed vibe-coding tasks on a purpose-built platform: replicate an app, add a feature, build an app from a text description alone. They prompted an LLM, watched the rendered behavior, and iterated. The source code was never visible. Human graders, blind to the other measures, scored the apps.
Both fundamentals predicted success. CS achievement correlated with vibe-coding performance at r = .39, writing skill at r = .29. In a joint model, CS knowledge contributed roughly twice the unique variance of writing. And CS knowledge stayed significant even after controlling for general reasoning ability. The authors’ own conclusion:
We found that both computer science achievement and written communication skills were positively and significantly correlated with vibe coding skills (r = .39 and r = .29, respectively), a correlation that persists in the former case after controlling for cognitive ability.
Thorgeirsson, Weidmann and Su, CHI 2026, p. 15 (Conclusion)
One more finding matters for this argument. Because the code was completely hidden in these tasks, the authors argue their estimate is conservative for ordinary AI-assisted work:
As such, our estimates for the role of CS achievement in general LLM-augmented programming should be viewed as a lower bound: in more permissive workflows, the same underlying skills can contribute both via prompt quality and via targeted edits to generated code.
Thorgeirsson, Weidmann and Su, CHI 2026, p. 14 (Result Generalizability)
Where the tweet oversells
The X post by @thesupermanmx is the version most people will see. It is punchy, mostly accurate, and gets the framing right in its best lines:
You can prompt your way past syntax. You cannot prompt your way past a fundamental lack of engineering intuition.
@thesupermanmx on X, September 5, 2026
But three of its framings do not survive contact with the paper. The study is from ETH Zurich, not China. The participants were university students in a lab, not the “100 developers” the post describes. And a study of task performance in a controlled environment cannot carry the weight the post puts on it, because the post is answering a different question than the one DHH was asked.
That scope difference is the whole story. DHH’s claim is about product-shaped work: building real software where deciding what to build, for whom, and in what order dominates. The study measures task success on timed, well-defined assignments with a grader rubric. Both can be true in their respective scopes. A student who knows CS will do better on a fifteen-minute build task than one who does not. That says little about whether a senior engineer’s instinct to prescribe a full architecture hurts them when the goal is an unstructured product problem. The paper never touches that question, and it never measures skill decline at all. Nobody “got worse” in this study.
There is also a practical gap the study’s design cannot close, and it is where most senior engineers actually live. Nobody senior “vibes” a production system: the working pattern is fleets of agents running under review, with a human in the loop reading diffs, rejecting plans and steering. That is implementation expertise applied continuously, not suspended. The study hid the source code from its participants on purpose, which is precisely the part of the workflow senior engineers refuse to give up. So even the narrow DHH claim describes a habit, not an inevitability, and the study’s setup excludes the population best placed to avoid it.
DHH’s claim survives, in its narrow form
Read carefully, the study nuances the strong “fundamentals no longer matter” narrative rather than contradicting DHH’s steelman of it. The version of the argument DHH was pushing back against is the one where implementation expertise becomes worthless once agents write the code. The data refuses that: even with the code hidden, CS knowledge was the dominant predictor of who built working software. If anything, the study hurts the people who used DHH’s headline as license to skip fundamentals.
Where it genuinely cuts against DHH is on emphasis. His account of his own experience implies implementation depth can be a liability that you have to manage. The study found no evidence that expertise hurts anyone. CS knowledge helped, full stop. What the study does share with DHH is the mechanism underneath both: the differentiating skill is judgment over what the agent produces, and that judgment needs something to be made of. Prompting fluency alone was worth less than domain knowledge in every model the authors report.
The paper also carries one erosion-flavored datum, and it is worth reporting honestly: heavier self-reported LLM use coincided with weaker essays and weaker vibe-coding apps. The authors themselves hedge it properly:
Lastly, we find it interesting that prior LLM usage is negatively and significantly correlated with written communication and vibe coding performance. We speculate that this relationship is caused either by (i) LLMs having a negative effect on students’ ability to express themselves or (ii) that students who are less proficient at writing are more likely to use LLMs, or (iii) a combination of both.
Thorgeirsson, Weidmann and Su, CHI 2026, p. 14 (Discussion)
That is a correlation with a plausible selection explanation, exploratory, not causal. If it survives better designs, it becomes the first direct evidence relevant to whether AI erodes skill. Today it is a hypothesis. Presenting it as proven would be exactly the kind of overshoot this field does not need more of.
What this changes for hiring and for you
The original piece said hiring criteria shift toward product sense and verification, without evidence. The study gives that claim its first empirical footing, with a caveat. If CS knowledge is the dominant predictor of AI-era building success, then teams that stopped testing fundamentals because “the AI writes the code now” are screening for the wrong thing. The verification argument from the original article still stands, and the study strengthens it: judgment over agent output is exactly what separates people in the data.
For the individual engineer, the actionable version is the one DHH hinted at and the study makes concrete. Your implementation knowledge is not the liability. The habit of prescribing a path when a described problem would do is. That is testable this sprint: hand one real task to an agent as a bare problem statement and another as your usual detailed spec, and compare outcomes. If your specs are not paying for themselves, you now have evidence instead of a podcast clip.
If your teams are building platform work with agents and you want an independent verdict on what they actually produced, that is the job I do: the data platform performance audit reviews the whole stack, and a free 15-minute intro call is the way to start.
The takeaway
A preregistered study found that computer science knowledge is the strongest predictor of vibe coding success, roughly twice the weight of writing skill, and it found no evidence that expertise hurts. That pushes back on the “fundamentals are dead” reading of DHH’s claim, which the study’s most viral summary on X presented as settled. But it does not touch DHH’s narrow, fair version: on product-shaped problems, the differentiating skills are deciding and judging, and they are unequally distributed. The data and the podcast are answering different questions. Confusing them is how both sides end up overclaiming.
The honest position after the first controlled data: fundamentals still compound, judgment still gates output, and nobody has shown that AI makes good programmers worse. AI just made the argument louder.
FAQ
What did the ETH Zurich vibe coding study find?
In a preregistered study of 100 university students (Thorgeirsson, Weidmann and Su, CHI 2026), computer science achievement was the strongest predictor of vibe coding performance at r = .39, writing skill followed at r = .29, and CS knowledge stayed significant after controlling for general reasoning ability. The study found no evidence that programming expertise hurts AI coding performance.
Does the study prove DHH wrong about AI coding?
No. The study measures task success on timed, well-defined assignments, while DHH’s claim concerns unstructured product work where deciding what to build dominates. It also never measures skill decline; nobody got worse in it. The data and the podcast answer different questions.
Do senior engineers actually vibe code production systems?
Not the way the study’s participants did. The common senior workflow is fleets of agents under continuous human review: reading diffs, rejecting plans and steering. The study deliberately hid source code from participants, which excludes the review-heavy workflow where implementation expertise is applied continuously.
Sources: Thorgeirsson, Weidmann and Su, “Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency”, CHI 2026; @thesupermanmx on X, September 5, 2026; prior article: Are programmers worse at AI coding?. The author has no affiliation with ETH Zurich or the X post’s author.
0 Comments