Where AI coding agents fail: six faults a green build hides

AI coding agents fail in six ways, and a green build hides all six. Three of them are work reported as done that never ran, an instruction read as more than was meant, and a narrow fix whose reason is not in the code. The other three are a change that compiles and does the wrong thing, a rule no single diff shows, and an architecture nobody chose.

The dangling promise: an action described, never taken

We first named this one on a client marketplace, in the customer-facing assistant. The assistant would end a reply with something like "just pulling those results together for you now". Nothing runs after a turn ends. The visitor sat watching a promise the system had no mechanism to keep.

A dangling promise is a turn that ends with an agent describing an action it never performed.

What makes this class nasty is that it is invisible everywhere you would think to look. The transcript reads perfectly. The logs record a successful response, because a message was returned. Nothing anywhere throws. The only party who knows something is wrong is the person waiting.

We have two defences and only one of them is a control. The prompt rule (the thinking is finished before you write, so present the outcome and never the intention) lowers the rate. The control is server side. After the agent loop, if the state says a search should have run and the message contains a promise, the server runs the tool itself and replaces the message with the real outcome. Prompts instruct, code enforces. That one behaviour has dedicated tests.

Coding agents do the same thing in a different costume. A turn ends with "now I'll run the test suite" or "next I'll update the other call sites", and then it just ends. Nothing ran. The summary and the work are two separate records, and only the work had to be true, so an audit trusts the diff and the test results over the summary on top.

Reading intent: when a casual sentence sounds like permission

Instructions reach an agent in ordinary English, and ordinary English is full of sentences that sound like permission without being any. A remark about what "we" will do once a review is finished is a plan to a colleague and can read as a go-ahead to an agent. A competent human reader resolves it from context the agent cannot weigh: who owns the repository, what the review was for, what a person does with their hands at the end of one.

An instruction in a prompt is one more sentence for the agent to weigh. So the actions that matter have to be ones it cannot take without a person approving them: pushing to the main branch, running a migration, reading a production key, deleting anything it cannot restore. Version-control writes and deployments reach outside the project, so they are the first actions to keep with a person. A good AI use policy spends more of its length on that list than on the approved tools.

The narrow fix in a wide codebase

Some of the hardest bugs are the smallest. The narrow fix needs the reason the code is odd, and the reason is usually not in the code. Take the scheduling bug that only appears on a fortnightly anchor date: the fix is tiny, and finding it takes an afternoon of knowing why the anchor exists. Nothing in the file says why the anchor is there, so nothing says which lines are safe to change. An agent reading it can mistake the fix for an oddity and tidy it away.

The reason lives in a conversation with a client or in the memory of whoever was there. An AI-built app hands over its repository without the conversation behind it. If the reasons were ever stated, they stayed in the AI app builder's chat history, so whoever inherits the app meets every odd line without its explanation. The familiar symptom is that every fix breaks something else, usually because there are no tests and the same rule has been copied across several screens.

Two habits narrow the gap. Write tests that pin today's behaviour before changing anything, so a fix that breaks a neighbour shows up the same day. Then leave a short decision record beside any line that looks wrong but is not, with the reason it is there.

Plausible and wrong: code that compiles and misses the point

The failure everybody names, and almost nobody defends against properly, is code that builds, passes lint, passes type checks, and does the wrong thing with total confidence.

What gets through the build includes an update endpoint that rebuilds a field it should preserve and the same request-header logic copied across every proxy route. It also includes a stale closure in a React hook and validation messages swallowed by the API client, so users see a generic error where the server sent a precise one. All of it compiles, and each change looks fine to a reviewer reading it on its own.

Review attention is the mechanism. Reading a short diff is easy, reading a diff that spans dozens of files is not, and a single prompt to an AI app builder can produce the second kind. Attention thins out at precisely the point the blast radius is widest. The same decay happens inside a batch: a check that was thorough on the first item does less work on the sixth, so it has to run on each piece of work.

The same rigour has to reach what an agent writes about its own work. A generated completion report can call a feature compliant with a regulatory regime the product does not operate under, and that claim stands until a person checks it. Whether a rule binds you turns on where your systems and their output land, a question worked through in whether the EU AI Act applies to UK companies. An agent is in no position to answer it about its own work.

Cross-cutting invariants that no single diff will show you

Some rules are invisible in any one change. Every query filtered by tenant. Every money value handled as a decimal and never a float. An agent asked to add a feature will honour the invariants it can see in the files it happened to open, which is not the same as honouring them.

So the control is a pass whose only job is checking the codebase against its own written architecture. The agent's judgement and a reviewer's memory cannot stand in for it, because a rule broken thinly across many files is exactly what no individual diff exposes. On one platform we build, a pass of that kind sorted its findings into four severity tiers, with data integrity at the top. The lowest tier was bringing the written architecture up to date with the code, which is the part people skip and the part that stops the drift coming back.

When an app has no written architecture to check against, a senior engineer running an AI-built app audit reads every path that money or personal data takes. The report ranks what it finds by what would hurt first.

Money arithmetic is the other family. For a client marketplace we build, the rule when importing supplier pricing is that a row whose price we cannot parse gets dropped, never defaulted to zero. A zero is a real price, and nobody spots it in a several-hundred-cell import. That went into the code as a projector that rebuilds the output from a fixed field list, so anything the model invented cannot reach the database. Unknown fields fail closed.

Either rule is simple to follow once it is written down. The missing rule is the one nobody notices, because an absence leaves nothing in a diff to flag.

Architecture calls, and why a model swap will not make them

The most expensive decision in an AI feature is not which model you pick.

In a client marketplace we build, we got tired of choosing models by feel and built an evaluation harness. It laid out model plus configuration plus engine as cells and ran scripted multi-turn scenarios with per-turn expectations, replayed against the real orchestration code. Eleven models went through it. Each turn recorded latency, token counts, computed cost, tool precision and recall, whether the output contract held, and a judge score. The judge came from a model in a different family than any contestant, so same-family self-preference stayed out of the numbers.

The models did differ, sometimes comically. One wrote by far the best copy and reached for its tools about 17% of the time, which is the wrong shape for an agent whose job is calling things. Another was fastest and cheapest and scored worst, because it emitted tool calls inside its reasoning trace. But the finding we still use is the dull one: an A/B between two flow engines, same model on both sides, moved the score further than any model swap did.

A model swap is a configuration change, which is why everybody runs that experiment and almost nobody runs the other one. Changing the shape of the system is expensive and open-ended, and it is what moved our numbers. Choosing an architecture means holding what the business will need in eighteen months against what the team can operate on a Tuesday, and neither of those is in the repository. In an AI-built app, the decision nobody made tends to surface as a data model that fights the product, where every feature takes longer than the last. Swapping the tool that generates the code is the same cheap experiment, and it leaves that decision unmade.

The winning configuration was held back from production anyway, because a secondary metric sat at 89% against a 95% gate. We had set that gate in advance, so the argument about waiving it would be short.

What the six have in common, and what to ask a supplier

What runs through all six is that an agent supplies reach and a person still has to supply judgement, and the two fail in opposite ways. Reach fails loudly. The build breaks, the tests go red. Judgement fails silently, in a transcript that reads well, a diff that compiles, a completion report nobody re-read. A demo that works proves little for that reason, because failures of judgement are exactly what a demo does not show.

For a founder working in an AI app builder, that reduces to three questions. If it goes wrong, will it announce itself or hide behind an output that looks fine? Is the rule this work has to respect written down where the tool will read it? And which actions can the tool take on its own?

A company buying from a supplier should ask two more, in writing: how the supplier controls its AI tools, and who reads each change before it merges. Quantum Craft's answer on supplier questionnaires reads: "AI tools are used only through business accounts whose terms bar training on client code. They never hold production credentials, and client personal data stays out of them. A senior engineer reviews every change, and automated tests run on every one."