AI made writing code cheap. It didn’t make reviewing it any cheaper — and on most teams, review was already the slower half of shipping, not the writing itself. Once one half of a two-part process gets faster and the other doesn’t, the slow half becomes the entire bottleneck. That’s not a new law of software delivery; it’s just far more visible now than it used to be.
Here’s the actual system I use, and want to keep building, for reviewing AI-written code. Some of this is already how I work. Some of it is where I’m borrowing a good idea and adapting it. I’ll be specific about which is which, because I think the honest version of this is more useful than a tidier one.
1. Match the review to the risk, not the diff
Not every change deserves the same scrutiny. A documentation fix and a change to payment logic carry entirely different risk profiles. Treating them identically means either over-reviewing the boring stuff or critically under-reviewing the dangerous stuff.
My target state is to have the AI tool itself dynamically tag the risk level on a merge request. I want to make that tag actionable — an actual label on GitHub or GitLab rather than a note hidden in a comment — so it can drive real engineering automation: required reviewer counts, branch protection rules, and CI gating tied directly to the tool’s classification. And if a change is both large and high-risk, the tool shouldn’t just throw more human reviewers at an unreadable diff; it should suggest splitting the PR into smaller, isolated changes before anyone reviews anything.
I know exactly why this matters because I’ve paid for not having it. A 50-file merge request on my team once shipped a quiet duplication bug — the same UI component copy-pasted across several sections instead of shared. It took months to surface because the diff was simply too massive for anyone to read carefully. More reviewers wouldn’t have fixed that. A smaller diff might have. (The full story is in The Review Metrics Most Teams Don’t Track.)
This instinct — sorting changes by risk to decide review depth — is close to what Anthropic’s own engineering team has described doing at a much larger scale. Worth reading if you want the enterprise-scale version of the same idea.
2. Contextual guidance: tell reviewers what matters right now
A generic review prompt treats every file the same way. But your codebase almost certainly has specific landmines — patterns that look fine on the surface but aren’t, or conventions that aren’t obvious from the isolated diff alone.
The fix I want to use is a context file that tells an AI reviewer what to actually pay attention to in my codebase — an idea close to what Anthropic’s engineering team has described using themselves. The extension I want to push further: move away from a single, static version of this file for the whole repository. Instead, the context should update per merge request, describing what is specifically worth scrutinizing in this change.
A repo-wide file tells a reviewer about your codebase in general. A per-change note tells them what is actually new, fragile, or risky about what they’re looking at right now — exactly the context missing when a human or an LLM opens a large, unfamiliar diff cold.
3. Run specialized review passes (and demand less prose)
A single, generic review pass on a merge request has a hard architectural limitation: it only sees what’s directly in front of it. It has no awareness of what three other PRs quietly introduced last week, which makes it blind to cross-PR duplication.
I run a security-focused review pass when a change looks suspicious — Claude Code ships a /security-review command for exactly this, and Anthropic’s own team describes an automated security review whose open findings block a merge. I want to add to this rotation: a cross-repo duplication check, and a dedicated “simplify this” pass. Left to its own devices, AI will often generate more defensive complexity than a problem requires; it’s worth forcing the tool to justify every layer of abstraction it creates.
The bigger shift I’m building toward is changing how the AI communicates back. Long paragraphs of AI-generated analysis are expensive to read carefully. Worse, the analysis sometimes changes meaningfully the moment you push back or ask a clarifying question, meaning the first version wasn’t as settled as its confident tone suggested.
Instead of forcing me to pull status updates, I want the agent to tell me, unprompted, in a short structured response, exactly what it just finished, what’s next, and where it’s blocked — done, next, blocked-on, not a wall of prose.
4. Build systemic triggers, not mental checklists
I’m still systematizing the automation of these passes, but the one piece of the framework I rely on whenever a change is headed to staging or production is asking for specific test scenarios up front. Not a generic “test this code” — named, concrete cases.
We recently resolved a production incident where a payment retry workflow had silently stopped processing for about a month, triggered by a single customer record with no usable card on file. The most valuable part of the fix wasn’t the code change itself; it was defining three exact test scenarios upfront: a card that declines but can retry, a customer with no card at all, and a control case.
Testing that control case — a perfectly good payment, run in the same batch as the broken one — proved that one bad record could no longer stall the wider pipeline. It told us more than either failure scenario would have told us in isolation.
Naming the exact scenarios to validate is the part of “making it a system” I trust most. But more of the rest is built than the opening of this section suggests. On the services my team owns, automation runs in CI/CD against staging, and every change has to bring its own automation in a separate merge request. Automated checks cover merge request titles and size, an AI review pass runs on every merge request, and we hold a high unit-test coverage bar with sanity checks after release. A dependency-update bot opens its own merge requests, a failed rollout rolls back automatically, and a flaky test gets one automatic retry.
Not everything worked. We tried auto-fixing lint on the merge request itself and dropped it, because it buried the real change under unrelated edits, which is the opposite of what a reviewable diff needs. Since most of our code is now AI-written and comes out formatted, that noise is mostly prevented at the source.
What I haven’t built is the always-on version. Auto-creating automation for new features is still an experiment, and auto-approval for lower-risk merge requests is what I’m building now.
5. Release gradually and measure what’s measurable
Anthropic’s Claude Code team has written about tuning review for the least human time that doesn’t raise reverts or incidents, with a classifier deciding which changes need a person at all. Most growing teams, mine included, don’t have that level of telemetry yet — and it’s worth describing what we actually do instead of pretending otherwise.
In practice, our safety net relies on gradual deployment and human validation: the engineer who owns the change tests it on staging, runs a sanity check in production right after deploy, and monitors error budgets and workflow-engine dashboards for anomalies. For high-impact features, we run a dogfooding pass or a bug bash with product managers and other engineers before calling it done. Some changes get extended monitoring — a week or more — depending on how much could actually go wrong.
It’s a less mathematically precise process than a fully instrumented metrics pipeline. It’s also, I suspect, much closer to how most engineering teams actually operate today. Acknowledging that gap is better than implying a level of rigor that isn’t there yet.
The actual point
None of this works if the goal is reading every single line of code an AI writes. Human teams can’t keep up with that volume, and pretending otherwise just means review quietly stops happening under deadline pressure, without anyone deciding that on purpose.
The point is matching real review effort to real risk, and building enough structure that doing the right thing doesn’t depend on a human remembering to. I’m not fully there on all five of these — naming exactly which parts are solid and which are still aspirational is more useful to another engineering leader than pretending the system is finished.
Further reading: The Review Metrics Most Teams Don't Track and the component-duplication incident go deeper on the risk-tiering story above. The Volume vs. Judgment post covers why review effort has to scale with risk instead of getting skipped under pressure.
Some of the structure here was informed by Anthropic's own published engineering practices: "Redesigning source control at Anthropic" and "How the Claude Code team does code review" by Addy Osmani, Morgan Westlee Lunt, and Qing Wang. Where the practices above are mine, they're mine; where they're borrowed and adapted, I've tried to say so plainly.
Team practices and incidents above are described in general terms. Details have been genericized to protect confidentiality; the failure modes and the lessons are real.