When the author and the reviewer of a change can both be machines, “keep a human in the loop” stops telling you what the human is for, or who answers when a machine-approved change breaks production. Pull requests on GitHub grew fivefold over three years, according to industry survey data. Your reflex is the right first question: which changes still need a human? Governance of machine review is four questions: routing, audit-proof policy, measurement, and build-versus-buy. This is the governance side of machine review.
How should I decide which changes need a human reviewer, and is full agent sign-off defensible for a regulated codebase?
Decide by blast radius, not lines changed: a 400-line change to an internal admin page is low risk, a four-line change to an interest calculation is high risk. Full agent sign-off is defensible only inside a bounded, evidenced scope a named human owns.
Risk-based PR routing is the mechanism: classify changes by consequence and route accordingly, using a blast-radius scan and data-driven risk assessment from historical incident data. As Rootly’s team put it, the size of the diff stopped being the useful signal. Inside a stacked workflow, risk-based routing has to work differently.
Three tiers cover most codebases: auto-approve where consequences are contained, agent review plus a human spot-check where consequences are moderate, human sign-off where a change is irreversible, regulated or customer-facing. Separate the three agent postures: assist produces commentary, gate produces a blocking check, sign-off carries approval authority, and only the third raises the accountability question.
Is full agent sign-off defensible in a regulated codebase? Only where the evidence is retained and someone owns the control. Meta landed 331,720 diffs with incident and revert parity (no worse than human-reviewed changes), after excluding SOX-scoped code, per the RADAR study. That is one organisation’s result, so treat it as a precedent rather than a guarantee. Under APRA’s April 2026 letter, “the model decided” is not an answer under CPS 230 and CPS 234, and the EU AI Act applies to European customers. Which raises the question routing cannot answer: who owns the control, in writing.
How do I write a review policy for AI-generated code that survives an audit, and who is accountable when an auto-approved change breaks something?
A policy survives an audit by being falsifiable: it must state what is auto-approved, on what evidence, retained for how long, and who owns the exception path. Accountability then defaults upward to the leader who authorised the control.
That means four properties: explicit change classification, with the auto-approve threshold stated as delegated authority; an evidence trail on every auto-approval; an exception path with an owner; and a review cadence that assumes revision. None of it works without review instrumentation: every auto-approval must record its classification, approval path and justifying evidence, as the agent-governance guidance frames it. Once rubber-stamping becomes the default, the evidence is what outlasts reviewer attention.
Accountability does not move. The engineer who merged is answerable, and the tooling gives the postmortem something concrete instead of “the AI wrote it”, as the attribution argument runs. Record the authorisation deliberately.
Two things keep it honest. Give the agent a distinct identity with scoped, revocable credentials bound to a person, because an agent that proposes and approves breaks segregation of duties; each agent needs its own identity. And treat the reviewer agent as production code: prompt and threshold changes travel the same review path.
Auditors map your policy onto SOC 2 Type II and ISO 27001 and want continuous operation across the audit period, not on-demand screenshots. Morgan Stanley runs deterministic tools with verification and auditability unattended, as reported here. The FINOS SDLC Governance Working Group is drafting the shared control language so you can cite a control rather than invent one. The test is whether your policy catches last quarter’s closest call without “we’d have to do it manually.” A written policy only matters if you can prove it is holding, which is what measurement is for.
What should we measure to know whether review is working, beyond PR count and cycle time?
Stop reporting throughput alone. A team that rubber-stamps improves both PR count and cycle time, so those numbers cannot certify that review is working. Measure outcomes instead: escaped defects normalised by change volume, coverage of high-risk changes, comment usefulness and rework rate.
The measure that holds is escaped defects, defects found in production rather than before merge, because it measures an outcome, not activity, and an agent cannot make it look better without producing a defect to answer for.
Add the review quality metrics: defect escape rate for high-risk classes, review coverage above the blast-radius threshold, comment usefulness, and rework rate, the share of PRs with more than two revision rounds, as tracked here. None of this is knowable without the review instrumentation described above: classification, approval path and post-merge outcome captured as one record, per the AI-native SDLC guidance.
These metrics approximate one thing: change confidence, a quantified judgement that a change is safe to merge. That is the numbers behind the board ask, feeding the wider accountability picture. None of this is measurable without the right machinery, which raises the final question: build it or buy it.
How do I decide whether to build internal review tooling or buy it?
Buy when the capability is commoditised and the workflow integration is the hard part. Build when your differentiator is your own risk model, your own blast-radius data, or a compliance requirement no vendor will meet. Then check pricing: per-seat and usage-based models scale differently as agent volume rises, and the cheaper one depends on changes-per-engineer per week, not headcount.
The build-versus-buy frame has four parts: differentiation, control, time-to-value and who carries the maintenance burden, as platform decisions are usually framed. If your risk model and blast-radius data are the moat, own the machinery; if integration is the hard part, buy it.
Per-seat pricing is flat, predictable and rewards broad deployment; usage-based pricing tracks volume, so it grows even when headcount is flat, as the category’s price pages show. At 100 engineers the cheaper model depends on changes-per-engineer per week, not seat count, and agent adoption pushes that ratio up, which is exactly when usage pricing becomes expensive.
Metered pricing also creates pressure to reduce review coverage, a governance problem disguised as a finance one, as the pricing analyses warn. When evaluating vendors, what matters is whether they can route on your blast-radius data and defend your auto-approve threshold. Buy the commodity half, build the differentiating half, and bring benchmark evidence to the case.
Governance of machine review is a chain of evidence. Routing, policy, measurement and tooling each link to the next, and every link terminates in a named human. Routing is the entry point: a blast-radius model is defensible once written as a falsifiable policy with recorded authorisation. Measurement is the honesty check, because speed metrics cannot prove the control is holding. The build-versus-buy choice hides the last governance question: how you pay for review decides how much review you keep. This is the complete overview of review governance distilled into its four questions.
Frequently Asked Questions
What is the difference between AI code review and the linters and static analysis we already run?
Linters and static analysis apply fixed rules and return a deterministic result, while an AI reviewer makes a judgement about intent, context and consequence. That difference is why only the agent raises an accountability question. Both belong in the same pipeline, but static analysis can gate a change without a human owner, and an agent that approves one cannot.
Does agent sign-off mean we can shrink the engineering team?
No. Agent sign-off changes where human attention goes, not how much accountability exists. The reviewers you keep are the people who own the routing model, sample auto-approved changes and answer for the control when it fails, and that work is skilled and ongoing. Teams that cut reviewer headcount before building that ownership discover the gap at the first incident, not before it.
Who approves changes to the review agent’s own rules, prompts and thresholds?
Treat the agent as production code with its own change control. Any edit to its prompts, policies or auto-approve thresholds should travel the same review path as a code change, with a named human approver and a recorded rationale. Without that discipline, the control you spent months authorising can be weakened by a quiet configuration commit nobody signed.
What happens if our review vendor silently upgrades the model behind the agent?
You lose the ability to claim you know how approvals are made. Ask vendors for version pinning, change notification and a revalidation window, then rerun a sample of historical changes against the new model before it touches production. A model swap is a control change, so the record should show the revalidation date, the sample size and who accepted the result.
What should an engineer do if they disagree with an agent’s approval?
They should be able to escalate, and the policy should say how. Give every engineer a single route to request human review of any agent-approved change, log the override and its reason, and review overrides weekly. A rising override rate is one of the earliest signals that the routing model is misclassifying a change class, and the log is also what an auditor will ask to see.
How do we stop the auto-approve boundary widening over time?
Review the boundary on a schedule against evidence. Each revision should state which change classes moved, why, and what the outcome data showed, authorised by the same named owner. Boundaries widen by increments under delivery pressure, so make every increment a recorded decision rather than an unread diff. Set the cadence quarterly, or sooner when an incident traces back to an auto-approved change.
How do we test a routing model before we trust it in production?
Run it in shadow mode first. Classify changes and record what the model would have approved while humans still decide, then compare those judgements against the incidents and defects that followed. Replay your last twelve months of changes against the proposed thresholds before you grant any agent real approval authority, and look hardest at the changes it would have waved through.
Do we still need CODEOWNERS files and named reviewers?
Yes, though the job narrows. Ownership still tells you who answers for a service, and that person is the natural authoriser of its blast-radius model and exception path. What changes is the volume: owners review the changes routing sends them, plus a sample of what the agent approved, rather than every diff by reflex. Routing, not reflex, decides how much of their week that takes.
Does risk-based routing apply to infrastructure and data pipeline changes too?
More so than to application code. A Terraform change to network rules or a schema migration can carry a wider blast radius than most feature work, and both are frequently irreversible once applied. Classify them by consequence the same way, and place infrastructure and pipeline changes in the human sign-off tier until outcome data justifies anything lower.
Do we have to tell customers or a regulator that AI approves some changes?
Not always, but assume you will be asked. Under APRA CPS 230 and comparable regimes, regulators care that the control is documented, evidenced and owned, rather than that a human clicked approve. If a customer contract or supplier questionnaire asks how changes are approved, the answer should be a policy reference, not a discovery.
What is the minimum viable version of this for a team of ten engineers?
Start with two things: a written list of what is auto-approved, and a log of every auto-approval with its classification and evidence. Ten engineers do not need tooling, but they do need a named owner and a monthly review of what the agent waved through. Write it on a page rather than a wiki, and revisit it when the first agent joins the pipeline.
