← All writing

Post-mortem

How many AI agents should review your code? I tried 99.

Aug 14, 2026

A property portal collected 25-agent, then 45-agent, then 99-agent review audits. The votes stacked into agreement. The safety came from somewhere much smaller.

Stacking more AI reviewers buys agreement, not safety. My property portal accumulated 25-, then 45-, then 99-agent review audits, and more votes did not make the release safer.

The app is 918TS, a public-facing property portal. Multi-agent review is the most convincing way I've found to feel rigorous while learning nothing new. The route that finally replaced my swarm is embarrassingly small: deterministic checks (same input, same verdict, every time) for privacy, access, and build, plus one independent critic when the stakes clear a bar. This is the story of the pyramid, and of what tore it down.

Why does adding AI reviewers feel safer than it is?

Because approvals look like evidence, and a swarm will manufacture approvals at any volume you request. Each added agent files another opinion. The opinions pile into something that reads like consensus, and consensus reads like safety. Not one step in that chain examines the release.

Nobody designs a 99-agent audit on purpose. You arrive by increments that each sound reasonable. Twenty-five reviewers, to be thorough. Forty-five when twenty-five felt thin. Ninety-nine when a release genuinely scared me. Every escalation felt like diligence while I was doing it. The ledger row that records the whole climb is blunter than I ever was: more votes did not make the release safer.

The mechanism is boring once you see it. Reviewers spun up from the same model families, the same prompts, and the same context share the same blind spots. Their errors correlate — they tend to be wrong in the same places, at the same time, for the same reasons. Voting is a tool for averaging away independent mistakes. Against shared mistakes, it does nothing except grow more confident.

What did 99 agents actually buy?

Agreement, at scale. That is the whole honest answer, and I have to be careful about everything past it. I never ran a controlled comparison: the same defect set pushed through the swarm and through the lean route, catch rates logged side by side. So you will not get a cost figure or a catch-rate table from me here, and you should distrust any post-mortem that happens to have both. The one sentence my operating ledger supports is the one I keep repeating: more votes did not make the release safer.

What the audits flagged, and what happened next, stays generalized here to protect the people involved. The shape is safe to state because it is the shape everywhere: the swarm was built to produce a verdict, and verdicts were never the scarce thing. A reviewer whose misses differ from everyone else's misses was.

The 99-agent audit was review theater. I get to say that harshly because I built the theater, sold myself the tickets, and sat through every show.

What replaced the swarm?

A floor, then a judgment call. The floor is deterministic checks covering the three failure classes that actually bite a public portal: privacy, access, and build. Deterministic means the same input gets the same verdict every time, no model judgment in the loop. These checks gate every release, and no quantity of agent approvals can outvote a failed one. One hundred approvals plus one failed privacy check equals a blocked release.

Above the floor sits one independent critic: a separate session with fresh context whose job is to break the release, not bless it. The critic is added only when the expected loss from a miss exceeds the review cost. Expected loss is the plain-words version of an insurance idea: how bad the miss would be, weighted by how likely it is. A change touching what the public can see of private data clears that bar instantly. A wording tweak on a static page never does, and gets no critic at all.

Why one critic instead of five? Independence is the ingredient the swarm never had, and it is hard to buy twice. A second critic drawn from the same model with the same briefing mostly re-votes the first. If I cannot make the next reviewer meaningfully independent of the last one, adding it buys me agreement again — and I have already been to that show.

The portal runs on the same philosophy as the rest of my system: "cannot fail silently" beats "cannot fail." The deterministic floor is the silent-failure alarm; the critic is judgment, rationed by stakes. The full tour of that system is at How do I know if AI is telling me the truth?.

When is one more reviewer worth it?

When you can name, in advance, a miss it would catch that your existing checks would not, and that miss costs more than the review. That is the entire rule. It is one sentence, and it retired 98 reviewers from my release path.

Notice what the rule never asks: how rigorous the process would look, how many perspectives it would add, how nervous the release makes you. Feelings grew the pyramid. The rule asks for a loss and a cost, and when both are small it lets the release ship on the deterministic floor alone.

So, the question for your setup: which review step exists because deleting it would feel reckless? If it vanished tomorrow, name the specific defect that would now reach production. If you cannot, you already know what that step has been producing.

I'm Ali — I run real life-and-work admin on AI agents, then check their work in the open. More at /about.

Published under my standards. Found an error? Tell me — corrections go in the corrections log, dated, never silent.

@online{ali2026ninetynine,
  author = {Ali},
  title  = {How many AI agents should review your code? I tried 99.},
  date   = {2026-08-14},
  url    = {https://alidoes.ai/ninety-nine-agent-review/}
}

Caught something I got wrong? Send it directly. Confirmed corrections go in the corrections log.