The Demo Worked. Who Signs Off on Production?

The demo ends with a useful answer. The room can see the potential. Someone asks how quickly it can go live.

I would put another question on the table: who is prepared to approve its use, for which purpose, on what evidence?

That question does not require months of paperwork. It requires clarity about the service being introduced and the consequences of relying on it.

An internal drafting assistant and an agent that changes customer entitlements deserve different levels of scrutiny. The release decision should reflect that difference.

Establish what is being approved

Scope needs to be specific enough to test. “Help the service team” could mean finding guidance, drafting a response, recommending an eligibility decision or updating a live record. Each gives the system a different role.

Agree on the intended users, permitted actions, information boundaries and unacceptable outcomes. Identify a business owner who accepts the service outcome, a technical owner who can operate it and the appropriate person to accept residual risk.

These responsibilities may sit with fewer people in a small organisation. They still need to be understood.

Build the production decision on evidence about purpose, access, evaluation, operations and recovery.

Test the cases that could change the decision

A useful evaluation set includes normal work, awkward exceptions and foreseeable attempts to misuse the service.

For a policy assistant, test an obsolete policy, conflicting sources, a question outside scope and a request for information the user cannot access. Test what happens when retrieval fails. Include malicious instructions embedded in retrieved material. Check whether the assistant acknowledges missing evidence rather than producing a plausible answer.

For an agent that acts, examine the action itself: was it authorised, correctly formed and recorded? Could a repeated request create a duplicate? Can the operator trace which evidence and tool calls led to the result?

Acceptance criteria need to match consequence. An overall average can conceal a small set of serious failures. Review failure types separately, including cases where the system should refuse, stop or refer.

The evidence should support a defined release scope. A successful test for one user group or one source collection does not automatically justify broader access.

Practise stopping the service

Ask the support team to demonstrate how it would pause the agent, revoke tool access, identify affected cases and move work to a human route.

A stop button is useful only when somebody knows when to use it and what follows. Recovery can involve restoring configuration, correcting a knowledge source, reversing an action where possible and contacting people whose cases were affected.

Keep evaluation, operational evidence, incident learning and change review connected throughout the service lifecycle.

Production also changes. A new model, a revised policy library, a different tool or expanded permissions can alter behaviour. Establish which changes trigger re-evaluation and who approves the resulting scope.

NIST's AI Risk Management Framework describes governance as a cross-cutting activity and risk management as continuous across the lifecycle. That is a useful discipline for the operating model, without pretending any framework can remove the need for judgement.

Before approving the next release, request a short evidence pack and a demonstration of the failure path. If the team can explain the limits, show the tests and recover the service, the approval conversation becomes much more concrete.