AI Said It Works. How Do You Know?

An AI coding tool can report success without ever having run your code. It often pattern-matches what a finished task looks like rather than executing anything and checking the result, so "done" and "tested" can both be guesses dressed up as facts. Magda Wiklak, another buildDay participant, described exactly this: "AI was sure the code was fully working. It didn't, spent lots of time finding out where the problem was." The AI's confidence and the code's correctness were two unrelated things, and she only found that out after the fact. This isn't a quirk of one tool. A 2026 study from Wharton researchers Steven Shaw and Gideon Nave found that 80% of people accept a confidently wrong AI answer, and those people rated their own confidence in the decision 11.7% higher than peers who worked without AI. The researchers call this cognitive surrender: the AI's confidence transfers to you, whether or not it's earned. A tool that sounds sure and a tool that is sure produce the identical sentence, which is exactly why the sentence alone can't be your evidence.

Developers say they distrust AI code and use it constantly regardless, because distrust without a fast alternative doesn't change behaviour. The Stack Overflow 2025 Developer Survey, published across December 2025 and February 2026, found only 29% of developers trust AI accuracy, down from 40% the year before, and 46% actively distrust it. At the same time, 84% use AI tools daily or weekly. The top complaint in the same survey was "almost right, but not quite," named by 66% of respondents. That phrase is the whole problem in miniature: code that compiles, looks plausible, and fails somewhere you weren't looking. Falling trust and rising usage aren't a contradiction. They describe developers who have stopped expecting the AI to be right and started treating every output as a draft that needs a specific kind of reading, not a blanket one. The rest of this guide is about what that reading looks like: which kinds of output need a close read, and which don't.

AI-generated code is safest in boilerplate, CRUD operations, and standard integrations with well-known libraries, and least safe in business logic, security-relevant code, and large diffs. Trust should scale down as the work gets more novel and more consequential, not stay flat across every task. This is the question Will, a third buildDay participant, put into words: "knowing when to trust the model knows how to do the job and when it needs more guidance." That's a skill you build by knowing where the failure rate actually concentrates, not by feel. CodeRabbit's analysis of 470 real-world pull requests, published 17 December 2025, put numbers on it. AI-generated PRs averaged 10.83 issues each against 6.45 for human-written PRs. Logic and correctness issues were up 75%, and security vulnerabilities were 1.5 to 2 times more frequent. Both concentrated in business logic and novel algorithms, not in boilerplate. A form component or a standard API wrapper is close to safe territory. A pricing calculation, an auth check, or a permissions rule is not, and deserves the read time that a form component doesn't.

Check whether the AI actually ran the code, whether the diff size matches what you asked for, and whether it invented an import or API that doesn't exist. These three are checkable in under a minute and catch most of what "it works" hides. Did it run anything, or just claim it? Simon Willison built two tools, Showboat and Rodney, specifically to close this gap, so that agents demonstrate their work with re-executable output instead of asserting it (simonwillison.net, "Introducing Showboat and Rodney, so agents can demo what they've built," 10 February 2026). If your tool can't show you the command it ran and the output it got, ask it to, before you take the claim on faith. One signal that does not work, despite feeling like it should, is hedging. It's tempting to read caveats and qualifiers as a sign of an honest, checkable answer. Research combining verbal hedging with confidence indicators found the opposite: it produced the highest rate of overreliance on wrong answers of any condition tested (ScienceDirect, 2026). A hedge is not evidence. Only re-executable output, a matching diff, and imports that actually resolve are.

Match your checking effort to the blast radius of being wrong, not to how long the AI took or how confident it sounded. A one-line config change and a 400-line refactor of billing logic are not the same decision, and treating them the same is where the risk actually lives. Larridin's "Verification Discipline" (larridin.com/blog, 26 July 2026) states the rule plainly: "A two-second approval on a one-line config change is fine. The same approval on a 400-line refactor of billing logic is a warning." That single line is the practical version of everything above it in this guide. In practice, that means asking one question before you approve anything: what happens if this is wrong and I don't catch it? If the answer is "nothing much," a glance is proportionate. If the answer touches money, auth, real user data, or anything hard to undo, read every line, and get the AI to show its work first, using the checks in the section above. The cost of being wrong sets the bar. Nothing else does.

No. This is the in-session judgement call that happens before either of those, not a replacement for them. Evals measure a finished feature's accuracy at scale, over time, against a test set and a grader: see our guide on AI evals for how to build that. Debugging is what you do once something is already broken: see the Reset Rule for when to stop prompting and revert instead. This guide sits in the moment before either applies: the AI has just told you something works, you haven't shipped it, and you're deciding whether to believe it. Get that moment right often enough and you need the eval pipeline less, and you hit the debugging spiral less often too. Addy Osmani frames the underlying stakes well. In "Comprehension Debt" (addyosmani.com, 14 March 2026), he writes: "Making code cheap to generate doesn't make understanding cheap to skip. The comprehension work is the job." And further: "Tests cannot answer that question. Only comprehension can." Checking the AI's claim, every time, at a level proportionate to what's at stake, is what that comprehension work looks like in practice.

How do I know if I should trust AI-generated code without checking it myself? Match your checking effort to how bad it would be if it's wrong. A one-line config change needs a glance. A change to billing logic, auth, or anything touching real data needs you to read every line. Trust scales with the cost of being wrong, not with how confident the AI sounds. Why does an AI coding tool say something works when it doesn't? Because it often hasn't run anything. Many tools declare "done" or "tests pass" based on pattern-matching what a finished task looks like, not on executed proof. Ask it to show you the actual command output, not just the claim, before you believe it. Is hedging language from an AI a sign I should trust it less? No, and treating it that way can backfire. Research on confidence signals found hedging language combined with confidence indicators produced the highest rate of people accepting wrong answers without checking. Hedging isn't a reliable trust signal either way. Check the work, not the wording. What kinds of AI-generated code are safest to trust without close review? Boilerplate, CRUD operations, and standard integrations with well-known libraries. Analysis of 470 real-world pull requests found AI-generated code carries 75% more logic and correctness issues and 1.5 to 2 times more security vulnerabilities than human-written code, concentrated in business logic and novel algorithms, not boilerplate. Is this the same as reviewing AI code with a formal evaluation process? No. Evals measure a finished feature's accuracy at scale over time, with a test set and a grader. This is the faster, in-session judgement call: whether to believe what the AI just told you before you've built anything worth formally evaluating.