Who Reviews the AI Reviewer?

An AI review adds useful evidence. Assurance also needs independent

An AI agent writes a change. Another AI reviews it. The tests pass. The team sees a clear route to faster delivery.

There is value in that workflow. There is also an assumption worth examining: how independent is the evidence that says the change is safe?

Two agents can share the same incomplete requirements, outdated architecture guidance or mistaken interpretation. Their agreement may tell us less than it appears to.

Give each check a different job

An AI reviewer can identify defects, question an approach and explain areas a human should examine. It is a useful contribution to review.

Other checks should supply evidence from different directions: tests derived from acceptance criteria, security analysis, permission checks, approved architecture rules and examination by someone who understands the business context.

Consider a change to an approval service. The code may compile and its new tests may pass, while the change quietly allows a user to approve their own request. A test suite written from the same mistaken understanding could reinforce the error.

Someone needs to bring the actual delegation rule into the review.

Combine AI review with checks that have distinct purposes and an accountable release decision.

The independence comes from the evidence, controls and responsibilities. Merely selecting another model does not establish it. A human reviewer also needs enough time, context and authority to challenge the change; a hurried approval adds little.

Protect the controls from the contributor

A contributor should not be able to redefine the conditions under which its own work is accepted.

For a coding agent, that means protecting the relevant permissions, required checks, branch rules, policy configuration and audit records. Changes to those controls need a separately authorised path.

Test files deserve particular care. Agents can help create and maintain them, but a proposed weakening of important acceptance tests must be visible to an independent reviewer. Otherwise the agent may solve a failure by changing what counts as success.

GitHub's documented controls for its cloud agent include a human review and merge requirement. That is a product-specific implementation; the wider architecture question is how each organisation protects its own release path and evidence.

Set authority by consequence

I would start with a narrow role and expand it when operating evidence supports the change.

An agent may first recommend a change for a human to implement. Later it may implement in an isolated environment and submit for review. Selected low-consequence actions may eventually qualify for bounded automation where permissions, verification and recovery are reliable.

High-impact actions still need the appropriate explicit approval. Progress through the lower-risk stages does not create a general entitlement to act.

Increase authority within proven limits, preserve approval for high-impact actions and reduce authority when evidence weakens.

The assessment should consider how far an error can spread, how easily the result can be checked and whether an action can be reversed. New or unfamiliar work deserves particular caution because past performance may say little about it.

I would also review how the reviewers perform. Use known examples of acceptable and unacceptable changes, inspect missed issues and update the process when business rules change.

At the next delivery review, trace one change from requirement to release. Ask what evidence would have caught a confident but incorrect interpretation. The answer will tell you more about assurance than the number of reviewers in the diagram.

Portrait of Edge Pereira

ABOUT THE AUTHOR

Edge Pereira

Enterprise & Solution Architect · AI Governance · Technology Strategy

I work with leaders to make sense of complex technology challenges and turn them into clear decisions, practical roadmaps and meaningful business outcomes. With over 25 years in enterprise architecture, technology strategy and transformation, my focus is on making architecture useful, AI adoption responsible and change achievable.