Most enterprise AI programs do not fail on model quality. They fail the first time someone asks: which agent did that, on whose authority, and can you prove it?
That question used to be hypothetical. It is not any more. Agents have moved from drafting text to taking actions — moving money, adjusting limits, touching patient records, filing claims. The moment an agent acts, it inherits every obligation that governs the human it replaced. Most stacks were never built to answer for that.
The category error
The prevailing mental model treats governance as something applied to a finished system: build the capability, then wrap it in policy, review and monitoring. That sequence works for models that produce outputs a human then acts on. It breaks completely for systems that act directly.
The reason is structural. When a human acts on a model's output, the human is the control. Their authority is established, their judgment is accountable, and the audit trail runs through decisions a person made and can be asked about. Remove the human from the loop and every one of those properties has to be reconstructed in software — or it simply does not exist.
Autonomy does not remove the obligations. It removes the mechanism that was quietly satisfying them.
This is why governance cannot be a later phase. The authority model determines what the agent is allowed to do, which determines what it can be built to do. Retrofitting it means rebuilding the agent.
Four layers, in dependency order
The work decomposes into four layers, and they have a strict dependency order that most programs discover the expensive way.
- Entitlements. An agent may only act inside an authority someone actually granted it — scoped, time-boxed, and traceable to a human principal. Without this, nothing below it means anything, because there is no reference standard against which an action is right or wrong.
- Policy enforcement at the point of action. A control that evaluates at login and never again is not a control. Enforcement belongs between the agent's intent and the system of record, where it can still refuse.
- Attestation and audit. Evidence of what happened and why it was permitted, retained on the reviewer's timeline. This depends on the first two: you cannot evidence conformance to an authority you never expressed.
- Model risk governance. Inventory, validation, challenge and monitoring — for a system whose behavior changes when a provider ships a new version.
Why the deadline moving does not help
Regulatory timelines slip. The EU's high-risk obligations moved. That reads as relief and functions as a trap, because the engineering does not get shorter — only the deadline moves, and the work is measured in architectural decisions rather than calendar weeks.
More to the point, the binding constraint in most jurisdictions is not new regulation at all. It is existing supervisory expectation applied to a new kind of system. A US bank does not need the AI Act to have a model risk problem with agents; it has a model risk regime and an examiner. A health system does not need new rules to have a minimum-necessary problem; it has the Security Rule. The obligations are already there, and they already apply.
What good looks like
The organizations that will be comfortable in eighteen months are not the ones with the most sophisticated models. They are the ones that can answer four questions about any action an agent took, on demand, from evidence: what authority permitted it, what policy evaluated it, what the system state was, and who was accountable.
If you can answer those four, the audit is a formality. If you cannot, no amount of model quality will save the program — because the failure will not be a bad output. It will be a good output nobody can account for.
This is not one jurisdiction's problem. A US bank answers those four questions to an examiner — and since the April 2026 revision expressly put agentic systems outside the model risk framework, the answer has to come from the bank’s own reasoning rather than a citation. An Indian enterprise answers them under the DPDP Act's purpose-limitation logic and the RBI's sharpening expectations for model-driven decisioning. A Gulf institution building at sovereign scale answers them because a national platform that cannot account for its agents is a headline risk before it is a compliance one. Different letterheads; the same four questions.
If you are standing up an agent program and cannot yet answer them, that is the conversation I have most weeks — compare notes with me.