There is a pattern I keep seeing in enterprise agent programmes, and it now has enough data behind it to name. A team stands up an agent. The agent needs to know things — policies, entitlements, product docs, case history, the last four years of tickets. The context window is a million tokens now, and the marketing says the model can hold a whole codebase, a whole case file, a whole quarter. So the team does the obvious thing: they stuff it. Everything the agent might need, every call. Retrieval becomes a formality. Curation becomes someone's job for later.

Call it context-stuffing. It feels like engineering. It is actually the deferral of three decisions — what the agent may know, what it may do with what it knows, and what you can prove about either — and every one of those decisions comes back with interest.

The bill arrives three times

The first bill is money, and it is the smallest one. Published list prices in July 2026 put a single full million-token call anywhere from $0.14 to $10 depending on the model tier — a 71× spread — and the pricing pages themselves now warn that large windows don't raise per-token cost so much as they raise runaway cost when nobody controls what goes in. An agent is not one call. An agentic workflow is dozens of steps, and a maxed window is re-read at every step. At mid-tier prices, a 20-step workflow carrying half a million tokens of "just in case" costs more per run than the analyst-hour it was meant to save (half a million tokens × $3 per million × 20 steps ≈ $30 a run). Teams discover this in the second month, in the CFO's office. This is the visible failure, which is why it gets all the attention.

The second bill is accuracy, and it is better documented than most teams know. The research on long-context degradation — Chroma's 2025 study ran 18 frontier models through it — shows accuracy dropping 30 to 50 percent well before the advertised window limit, with the familiar lost-in-the-middle failure: material buried mid-context loses more than 30 percent retrieval accuracy against the same material placed at the edges. The million-token window is real. The million-token attention is not. Context-stuffing doesn't give the agent more knowledge; it gives the agent more places to look away from the thing that mattered.

The third bill is the one that ends programmes. Here is the question that gets asked after an agent does something wrong — moves the money, adjusts the limit, touches the record it shouldn't have: what did the agent know when it acted, and on whose authority did it know it? A team that context-stuffed cannot answer. Not "answers slowly" — cannot, structurally. When you flatten your document stores, wikis, case systems and entitlement-scoped databases into one undifferentiated prompt, you have built an index under different permissions than the ones the agent inherited. The access-control decisions your organisation spent a decade encoding are not in the window. They were lost at index time. Every call, your agent knows everything anyone ever loaded, and the audit trail says only: the context was large.

The context-stuffing trap Stuffing the context window is three failures on one curve — and only the first one is visible on an invoice. 10K 100K 500K 1M tokens context loaded per call → cost per run re-read at every step, ×20 steps per workflow task accuracy −30–50% before the advertised limit (context-rot studies, 18 frontier models) provability can you show what the agent knew, and on whose authority? most agent programmes operate here max spend · degraded accuracy · no evidence The fix is not a smaller window — it is context assembled per call, from entitled sources, with provenance. vikramjha.work

The public version of this gap is already on the record. When two frontier models breached Hugging Face's production infrastructure this July, what made the incident reconstructable at all was that both companies could inventory what the agent's identity could touch and trace seventeen thousand actions across a weekend. Most enterprises running context-stuffed agents could not produce that reconstruction — not because their logging is worse, but because “what could it see” was never a decision anyone made. An agent that knows everything anyone ever loaded is the knowledge version of a credential scoped to everything: it does not care who finds it. (The full teardown of that incident is on the site.)

The mainstream failure, not the edge case

Gartner's poll of 3,400 organisations investing in agentic AI predicts over 40 percent of these projects will be cancelled by the end of 2027 — and the named causes are escalating costs, unclear value, and inadequate risk controls. The 2026 surveys underneath are bleaker in the middle: only 11 to 14 percent of agent pilots reach production at scale, and around half stall in the three-to-nine-month window after an apparently successful pilot. That window is precisely where the three bills land — the cost curve becomes visible, the accuracy sag becomes measurable, and someone senior asks the provability question for the first time.

The cancellations are being read as an AI failure. Look closer and they are a governance failure wearing an AI costume. The model did what it does. The programme failed because what the agent knew, and what it was allowed to do, were never designed — they were defaulted.

What the alternative actually is

The fix is not a smaller window and it is not better prompt discipline. It is an architecture — and the parts have to arrive in dependency order, because each layer defines what the next one is allowed to mean.

  1. Entitlements — a real answer to what may this agent know and do, granted by someone with the authority to grant it, scoped tighter than the human it replaced. The agent doesn't get tired, so it will exercise every permission it has, every time.
  2. A governed context layer — a context graph with permission-aware retrieval: the agent's knowledge assembled per call, from sources it is entitled to, carrying provenance on every hop. This is where context-stuffing is undone: the window gets small again because it holds only what survived a permission check.
  3. Policy enforcement at the point of action — not a PDF the agent has "read"; a gate the action passes through.
  4. Attestation — the record that lets you reconstruct, months later, what the agent knew, from where, under which entitlement, when it acted. The thing that turns the auditor's question from a programme-ending event into a lookup.

Skip layer one and layer two is just RAG with better marketing. Skip layer two and layers three and four are enforcing policy over inputs nobody controlled. This ordering is the whole game, and it is why governance cannot be a later phase: the authority model determines what the agent can safely be built to do, not just what it is permitted to do afterwards.

Governance is an architecture, not a phase Four layers, in dependency order. Each one defines what the next is allowed to mean. 1 · Entitlements What may this agent know and do — granted by someone with the authority to grant it, scoped tighter than the human it replaced. The agent exercises every permission it has, every time. 2 · Governed context A context graph with permission-aware retrieval: knowledge assembled per call, from sources this identity is entitled to, provenance on every hop. The window stays small — by design. 3 · Policy enforcement at the point of action Not a document the agent has "read" — a gate the action passes through, with the authority check made at act time, not at index time. 4 · Attestation Reconstruct, months later: what the agent knew, from where, under which entitlement, when it acted. Turns the auditor's question from a programme-ending event into a lookup. context-stuffing deletes this layer flatten it all into one window and layers 3–4 enforce policy over inputs nobody controlled Skip a layer and everything above it is theatre. This is why governance cannot be a later phase. vikramjha.work

Brussels blinked. The obligations didn't.

The strongest counterargument deserves a straight answer. Brussels just handed everyone an excuse to relax: the EU AI Act's high-risk obligations, due 2 August 2026, were deferred in late June — standalone high-risk systems to December 2027, AI embedded in regulated products to August 2028. If the deadline moved, why not defer the governance too?

Because the deadline was never the reason. The obligations that bind an agent moving money or touching a patient record didn't arrive with the AI Act and didn't leave with the omnibus — they came with SR 11-7, with HIPAA, with the DPDP Act, with the banking and insurance supervision that already governs the human the agent replaced. The moment the agent acts, it inherits those obligations whole. The regulator gave you eighteen more months of runway on one statute. Your first reportable event is not on that calendar. A deferral is not a reprieve; it is an interval in which the evidence layer is either built or assumed.

What good looks like

It is boring, and that is the point. An agent whose context is assembled per call from entitled sources. A window that stays small because it holds only what survived a permission check. An action log that answers the auditor's question in minutes. A cost curve that scales with the task, not with the archive. None of this demos as well as a million tokens of everything. All of it is still standing when the question arrives.

The window will keep getting bigger. That is exactly why the discipline has to come from your architecture — because it is no longer coming from the constraint.

I design this layer — permission-aware context graphs, entitlements, policy enforcement, attestation — for enterprises in North America, the Gulf and India putting agents into production that has to answer for itself. If the governance in your programme was assumed rather than designed, compare notes with me.