Most enterprise AI programmes do not fail on model quality. They fail the first time someone asks: which agent did that, on whose authority, and can you prove it?
That question used to be hypothetical. It is not any more. Agents have moved from drafting text to taking actions — moving money, adjusting limits, touching patient records, filing claims. The moment an agent acts, it inherits every obligation that governs the human it replaced. Most stacks were never built to answer for that.
The category error
The prevailing mental model treats governance as something applied to a finished system: build the capability, then wrap it in policy, review and monitoring. That sequence works for models that produce outputs a human then acts on. It breaks completely for systems that act directly.
The reason is structural. When a human acts on a model's output, the human is the control. Their authority is established, their judgment is accountable, and the audit trail runs through decisions a person made and can be asked about. Remove the human from the loop and every one of those properties has to be reconstructed in software — or it simply does not exist.
Autonomy does not remove the obligations. It removes the mechanism that was quietly satisfying them.
This is why governance cannot be a later phase. The authority model determines what the agent is allowed to do, which determines what it can be built to do. Retrofitting it means rebuilding the agent.
Four layers, in dependency order
The work decomposes into four layers, and they have a strict dependency order that most programmes discover the expensive way.
- Entitlements. An agent may only act inside an authority someone actually granted it — scoped, time-boxed, and traceable to a human principal. Without this, nothing below it means anything, because there is no reference standard against which an action is right or wrong.
- Policy enforcement at the point of action. A control that evaluates at login and never again is not a control. Enforcement belongs between the agent's intent and the system of record, where it can still refuse.
- Attestation and audit. Evidence of what happened and why it was permitted, retained on the reviewer's timeline. This depends on the first two: you cannot evidence conformance to an authority you never expressed.
- Model risk governance. Inventory, validation, challenge and monitoring — for a system whose behaviour changes when a provider ships a new version.
Why the deadline moving does not help
Regulatory timelines slip. The EU's high-risk obligations moved. That reads as relief and functions as a trap, because the engineering does not get shorter — only the deadline moves, and the work is measured in architectural decisions rather than calendar weeks.
More to the point, the binding constraint in most jurisdictions is not new regulation at all. It is existing supervisory expectation applied to a new kind of system. A US bank does not need the AI Act to have a model risk problem with agents; it has SR 11-7 and an examiner. A health system does not need new rules to have a minimum-necessary problem; it has the Security Rule. The obligations are already there, and they already apply.
What good looks like
The organisations that will be comfortable in eighteen months are not the ones with the most sophisticated models. They are the ones that can answer four questions about any action an agent took, on demand, from evidence: what authority permitted it, what policy evaluated it, what the system state was, and who was accountable.
If you can answer those four, the audit is a formality. If you cannot, no amount of model quality will save the programme — because the failure will not be a bad output. It will be a good output nobody can account for.