Where AI coding agents fail us: six tasks we still do by hand

We ship production software with coding agents every day, which is the only reason we can be precise about where they fail. Six task types still come back to a human every time: work an agent claims to have done, anything that turns on reading intent, the narrow fix, the plausible-and-wrong change, cross-cutting invariants, and architecture.

The dangling promise: an action described, never taken

We first named this one on a client marketplace, in the customer-facing assistant rather than in a coding agent. The assistant would end a reply with something like "just pulling those results together for you now". Nothing runs after a turn ends. The visitor sat watching a promise the system had no mechanism to keep.

A dangling promise is a turn that ends with an agent describing an action it never performed.

What makes this class nasty is that it is invisible everywhere you would think to look. The transcript reads perfectly. The logs record a successful response, because a message genuinely was returned. Nothing anywhere throws. The only party who knows something is wrong is the person waiting.

We have two defences and only one of them is a control. The prompt rule (the thinking is finished before you write, so present the outcome and never the intention) lowers the rate. The control is server side: after the agent loop, if the state says a search should have run and the message contains a promise, the server runs the tool itself and replaces the message with the real outcome. Prompts instruct, code enforces. That one behaviour has two dedicated test files.

Coding agents do the same thing in a different costume. A turn ends with "now I'll run the test suite" or "next I'll update the other call sites", and then it just ends. Nothing ran. So we read the diff and not the transcript. Of the two, only the diff had to be true.

Reading intent: when a casual sentence sounds like permission

On a field operations platform, a colleague ended a review request with "then we can push it". The agent read that as authorisation, committed, and pushed to main. It was not authorisation. "We" meant the human, later, after the review.

Nothing broke and the change was fine, which is the least interesting part of the story. The interesting part is that the sentence is genuinely ambiguous, and a competent human reader would have resolved it from context the agent had no way to weigh: who owns the repo, what the review was for, what a person does with their hands at the end of a review.

The rule that came out of it is blunt. Version-control writes and cluster operations are outward-facing actions the human keeps, always. Read-only invocations included, because "just check the status" sits one flag away from something that is not read-only. The rule overrides any instruction elsewhere that tells the agent to run git.

That correction lives in a feedback file: a short process rule in a name, description, why and how-to-apply format, loaded every session. Seven of them exist on that project, alongside seventy-four technical memory files. Technical memory records what the codebase is; feedback files record how we work, and every one of them exists because something went sideways once. The same thinking sits behind our AI use policy for a 30-person company. The useful section is not the list of approved tools. It is the list of actions that stay with a person.

The narrow fix in a wide codebase

Count commits on a healthcare platform we build and the ratio looks decisive. 58 of 117 carry an AI co-author trailer, and those commits moved about two-thirds of the code. On its own the number says nothing about who does what here.

The shape of the split says everything. The widest commits are all agent-driven. A 77-file front-end overhaul. A 66-file simplification that removed 4,073 lines. A 51-file repositioning pass. The human commits are narrow and late, two files or sometimes two lines, with messages like "calendar update" and, on one memorable occasion, "YEEEET".

Nobody decided this. It settled. Stating a rule once and applying it consistently across 77 files is something machines do better than people do at six in the evening. The narrow fix needs the reason the code is odd, and the reason is usually not in the code: the scheduling bug that only appears on a fortnightly anchor date, where the fix is two lines and finding it takes an afternoon of knowing why the anchor exists.

So breadth is a signal to delegate and subtlety is a signal not to. We are not sure that holds for long, mind. It may just be that the narrow fixes are the ones we have not got round to writing down yet.

Plausible and wrong: code that compiles and misses the point

The failure everybody names, and almost nobody defends against properly, is code that builds, passes lint, passes type checks, and does the wrong thing with total confidence.

A conformance pass on one platform turned up a decent sample in a single sitting: an update endpoint that rebuilt a field it should have preserved, the same request-header logic copied across fifteen proxy routes, a stale closure in a React hook, and validation messages swallowed by the API client, so users saw a generic error where the server had sent a precise one. All of it compiled. All of it had passed review at the time.

Review attention is the mechanism. Reading a 12-line diff is easy, reading a 77-file diff is not, and the widest diffs on our projects are all the agent-driven ones. Attention thins out at precisely the point the blast radius is widest. We found the human version of the same curve on a field operations platform: eight user stories shipped in one session, each of which had passed its clarification gate individually. Nobody skipped a step. The gate was simply doing less work by the sixth story than by the first, and the correction we filed afterwards moved the gate off the session and onto each piece of work.

The same rigour has to reach what an agent writes about its own work. On one platform, an agent-authored completion report described a feature as compliant with a regulatory regime the product does not operate under. Nobody lied. Nobody checked either. Whether a rule binds you turns on where your systems and their output land, a question worked through in whether the EU AI Act applies to UK companies. An agent is in no position to answer it about its own work.

Cross-cutting invariants that no single diff will show you

Some rules are invisible in any one change. Every query filtered by tenant. Every money value handled as a decimal and never a float. An agent asked to add a feature will honour the invariants it can see in the files it happened to open, which is not the same as honouring them.

So the control is neither the agent's judgement nor a reviewer's memory. It is a pass whose only job is checking the codebase against its own written architecture. On one platform we build, the architecture is written down in a form a machine can read, and we had an agent audit the code against it: 42 files touched in a single commit, findings sorted into four severity tiers with data integrity at the top. One was an isolation defect that no individual diff would have exposed. Caught pre-release, fixed. The lowest tier of that audit was updating the agent's own briefing files, which is the part people skip and the part that stops the drift coming back.

Money arithmetic is the other family. On a marketplace we build, the rule when importing supplier pricing is that a row whose price we cannot parse gets dropped, never defaulted to zero, because a zero is a real price and nobody spots it in a several-hundred-cell import. That went into the code as a projector that rebuilds the output from a fixed field list rather than validating whatever came back, so anything the model invented cannot reach the database. Unknown fields fail closed.

The agent implements either rule fine once it exists in writing. It has never once told us a rule was missing.

Architecture calls, and why a model swap will not make them

The most expensive decision in an AI feature is not which model you pick. We proved that on ourselves.

On a marketplace we build, we got tired of choosing models by feel and built an evaluation harness: model plus configuration plus engine as cells, scripted multi-turn scenarios with per-turn expectations, replayed against the real orchestration code. Eleven models went through it. Each turn recorded latency, token counts, computed cost, tool precision and recall, whether the output contract held, and a judge score from a model in a different family than any contestant, so same-family self-preference stayed out of the numbers.

The models did differ, sometimes comically. One wrote by far the best copy and reached for its tools about 17% of the time, which is the wrong shape for an agent whose job is calling things. Another was fastest and cheapest and scored worst, because it emitted tool calls inside its reasoning trace. But the finding we still use is the dull one: an A/B between two flow engines, same model on both sides, moved the score further than any model swap did.

A model swap costs one line of configuration, which is why everybody runs that experiment and almost nobody runs the other one. Changing the shape of the system is expensive and open-ended, and it is what moved our numbers. Agents execute an architecture well. Choosing one means holding what the business will need in eighteen months against what the team can operate on a Tuesday, and neither of those is in the repository.

The honest footnote: the winning configuration was held back from production anyway, because a secondary metric sat at 89% against a 95% gate. We had set that gate in advance, specifically so the argument about waiving it would be short.

What the six have in common

None of this is an argument against coding agents. We would not ship what we ship without them, and on one platform roughly two-thirds of the code churn arrived through commits with an agent co-author.

What runs through all six is that agents give us reach and we still supply judgement, and the two fail in opposite ways. Reach fails loudly. The build breaks, the tests go red. Judgement fails quietly, in a transcript that reads well, a diff that compiles, a completion report nobody re-read.

So the discipline is not a rule about AI. It is a question asked before handing anything over. Is this wide or narrow, because a mechanical sweep is a good delegation and a subtle two-line fix is not. If it goes wrong, will it announce itself, or sit quietly behind an output that looks fine? And the one people forget: is the rule this work has to respect written down anywhere a machine can read it? If it isn't, then writing it down is the job, and the job is ours.

We keep expecting that list to get shorter. Over the past year the wide sweeps have got much better. These six have barely moved.