Audit-evidence checklist
What a reviewer actually asks for, and the artifact that satisfies each ask.
Why it exists
Teams prepare for audits by writing policy. Reviewers ask for evidence. The gap between those two is where examinations go badly.
What it contains
The reconstruction test — can you show what a specific agent did on a specific date and why it was permitted
Authority evidence — the grant, its scope, and the human who issued it
Policy evidence — which policy version evaluated the action, and its decision record
Change evidence — model and prompt versions in effect at the time of the action
Oversight evidence — what a human reviewer saw, and how long they had to see it
Retention mapping — how long each class of evidence must survive, per regime
Third-party evidence — contracts, audit rights, version notices and tested exit paths
Containment evidence — circuit-breaker triggers, kill-path exercise and incident ownership
No email required, and nothing is recorded when you download. Use it, adapt it, argue with it — attribution is welcome, not a condition.
Evidence register
| Control | Question | Evidence that closes it | Common false positive |
|---|---|---|---|
| System boundary | Can you name the agent, owner, purpose, action classes, tools, models and production environments? | Approved inventory record with version and review date | The inventory lists a chatbot but not the actions or tools it can invoke |
| Human authority | Which accountable human or body granted each action class? | Grant record linked to agent and action class | A product owner is named but no authority was recorded |
| Delegation chain | Can authority be traced from principal through services to the executing agent? | Machine-readable delegation chain | The service account is treated as the authority |
| Scope | Are resource, value, rate, data-classification and jurisdiction limits explicit? | Bounded grant with enforceable constraints | Limits exist only in a prompt or policy document |
| Expiry | Does authority expire or require periodic re-approval? | Validity window and review event | Access is granted once and remains indefinitely |
| Policy decision | Which policy version evaluated the action before it took effect? | Decision record with policy identifier and result | Only the deployed application version is logged |
| Enforcement | Could the policy layer refuse the exact action in flight? | Allow/deny test and blocked-action evidence | Monitoring alerts after the system of record changes |
| Source of truth | Were material commitments checked against an authoritative source? | Lookup result, source version and comparison outcome | The model was prompted to follow policy |
| Action record | What happened, to which resource, when and with what outcome? | Immutable action event with correlation identifier | A chat transcript is the only record |
| Inputs | Can the material inputs and retrieved evidence be reconstructed? | Input references, retrieval results and hashes | Context disappeared when the session ended |
| Model and prompt | Which model, prompt, tool and configuration versions were effective? | Pinned identifiers on the action record | A moving provider alias such as latest |
| Human oversight | What did the approver see, decide and have time to assess? | Presented context, decision, identity and timestamp | A workflow status says approved without the reviewed material |
| Validation | Did testing cover authority, tools, failure containment and customer impact? | Independent validation plan, results and limitations | Only model accuracy or red-team prompts were tested |
| Change trigger | Which changes force revalidation? | Trigger matrix covering model, prompt, tool, data and policy changes | Provider updates do not create an internal event |
| Monitoring | Are thresholds tied to action risk and business impact? | Metric definitions, thresholds, alerts and disposition records | A generic quality dashboard with no action context |
| Containment | Can a bad agent be stopped before more actions land? | Tested circuit breaker and kill-path exercise | The response is to deploy a code fix |
| Third parties | Do contracts provide visibility, audit rights, version notice and exit? | Executed clauses and tested replacement or cease-use path | A vendor security page is treated as evidence |
| Incident response | Who owns an agent-caused event and how is evidence preserved? | Runbook, severity model, contacts and exercise record | The AI team assumes security will handle it |
| Retention | Does each evidence class survive for the applicable regulatory period? | Retention schedule mapped to systems and legal holds | Default application-log retention is assumed sufficient |
| Reconstruction test | Can an independent reviewer reproduce why one historical action was permitted? | Completed sample with evidence links and reviewer sign-off | The team explains what should have happened |
How to use it
- Select one material historical agent action.
- Link evidence; do not paste explanations into the evidence column.
- Mark missing, indirect or unreconstructible evidence as a gap.
- Have someone outside the delivery team perform the reconstruction test.
- Record the remediation owner, due date and retest result.
Built from public standards and general practice. The instruments cited in this artifact are checked against their primary sources on an ongoing basis. Instruments move — several cited here changed inside the last year — so verify against the source before you rely on one. If you find something stale, tell me and I will correct it.
Related: the other artifacts · the diagnostic.
Want this applied to your estate?
These artifacts are general by design — and they are the method behind the Agent Estate Review: two to three weeks establishing what is actually running, what each thing is permitted to do, and where you could not evidence it if you were asked next week. You have just read how it works.