# Utilization-Review AI Evidence Checklist

Version 1.0 — 4 August 2026
Vikram Jha — Defensible AI — https://vikramjha.work

Built on Maryland Insurance Article **§ 15-10B-05.1**, added by Chapter 747 of 2025 (House Bill 820, *Health Insurance – Utilization Review – Use of Artificial Intelligence*), approved 20 May 2025, effective 1 October 2025. Statutory language quoted from the enacted chapter text.

Maryland is used as the spine because it is the most specific enacted text in the United States on AI in utilization review. California, Arizona, Nebraska and Texas impose overlapping obligations, and at least twenty-five states have issued guidance on the NAIC model bulletin. The evidence a system must be able to produce is broadly the same; only the filing surface differs.

**This is not legal advice.** It is an engineering checklist for producing evidence. Verify every citation against the primary source on the day you use it, and take the legal question to counsel.

---

## Why this exists

Maryland defines artificial intelligence, for this purpose, as:

> "an engineered or machine-based system that **varies in its level of autonomy** and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that **can influence physical or virtual environments**."

Those two clauses describe an agent. The obligations below therefore attach to agentic systems by construction — at the same time as the April 2026 revised US interagency model risk guidance placed generative and agentic AI expressly outside its scope. The controls are being specified by state insurance law, not by federal model-risk supervision.

---

## Part A — Scope. Do these obligations bind you?

Answer all three. Any yes means the duties in Part B apply.

| # | Scope question | Source |
|---|---|---|
| A1 | Does the carrier use an artificial intelligence, algorithm or other software tool for the purpose of utilization review? | § 15-10B-05.1(B)(1)(i) |
| A2 | Does the carrier **contract with or otherwise work through an entity** that uses such a tool for utilization review? | § 15-10B-05.1(B)(1)(ii) |
| A3 | Is the entity a **pharmacy benefits manager or private review agent** that contracts with a carrier to provide utilization review on its behalf, and uses such a tool to do so? | § 15-10B-05.1(B)(2) |

**Note for vendors and service providers.** A3 binds you directly. A2 binds your client because of you. The statute does not stop at the payer, and a contractual flow-down is not the same thing as the obligation itself.

---

## Part B — The eleven duties, and the evidence each requires

Subsection (C) requires an entity in scope to *ensure* each of the following. For each: what the statute requires, the evidence that satisfies it, and a test you can actually run.

### (C)(1) — Determinations rest on the individual

**Requires:** the tool bases determinations on the enrollee's medical or other clinical history, the individual clinical circumstances as presented by the requesting provider, or other relevant clinical information in the enrollee's record.

**Evidence:** for a sampled determination, the record of which enrollee-specific inputs were retrieved and actually placed in the model's context, each traceable to a source record and timestamp.

**Test:** take one determination and list every enrollee-specific input the tool saw. "The system had access to the chart" is not an answer — access is not use, and a reviewer will ask which records were read.

### (C)(2) — Not driven solely by group data

**Requires:** the tool does not base its determinations solely on a group dataset.

**Evidence:** a decomposition of determination inputs into enrollee-specific versus population, actuarial and criteria-set sources, with a demonstration that the enrollee-specific inputs change the output.

**Test:** run the same determination twice — once with the enrollee-specific inputs, once without. **If the output does not change, the determination is being made by the group data.** This ablation is the only version of this test that produces evidence rather than assertion.

### (C)(3) — Criteria conform to the subtitle

**Requires:** the criteria and guidelines used for making determinations comply with the requirements of the title.

**Evidence:** a version-controlled mapping from each criterion the tool applies to the requirement it satisfies, with the version in force on each determination date.

**Test:** for a determination made six months ago, produce the exact criteria version then in effect. If criteria are edited in place, this is unanswerable.

### (C)(4) — The provider's role is not replaced

**Requires:** the tool does not replace the role of a health care provider in the determination process under § 15-10B-07.

**Evidence:** per determination — what the tool produced, what the reviewing provider was shown, what the provider decided, when, and whether the decision differed from the tool's output.

**Test:** report your override rate with timestamps. A review role that is never exercised, or exercised in seconds at volume, is a role in name. Median reviewer dwell time is the number that settles this.

### (C)(5) — No unfair discrimination

**Requires:** use of the tool does not result in unfair discrimination against enrollees prohibited by federal or state law.

**Evidence:** outcome-disparity analysis across protected characteristics, with the method documented, run on a defined cadence, and the action taken on findings.

**Test:** produce the most recent disparity analysis and say what changed because of it. "We do not use protected attributes as inputs" is not a finding — proxies exist, and the statute addresses results, not inputs.

### (C)(6) — Fair and equitable application

**Requires:** the tool is fairly and equitably applied, including in accordance with applicable regulations and guidance issued by the federal Department of Health and Human Services.

**Evidence:** a record of which HHS regulations and guidance were assessed as applicable and when, the conformance position taken, and evidence that the tool is applied consistently across lines of business and populations.

**Test:** is the tool applied to some populations, products or service lines and not others? If so, is that difference documented and justified, or is it an artifact of rollout sequencing?

### (C)(7) — Open to inspection

**Requires:** the tool is open to inspection for audit or compliance reviews by the Commissioner.

**Evidence:** a named, exercisable inspection package — what an examiner receives, in what form, within what period, and who owns producing it. Rehearsed at least once.

**Test:** request the inspection package today, giving five business days. **Did it exist, or was it assembled?** A standing inspection right means the evidence is constructed before the request, not after it.

### (C)(8) — Filed policies describing use and oversight

**Requires:** written policies and procedures are included in the utilization plan submitted under § 15-10B-05, including **how the tool will be used and what oversight will be provided**.

**Evidence:** the current filed plan section, plus a dated delta review comparing it against how the system actually behaves.

**Test:** read the filed description aloud to an engineer who built the system and ask whether they recognise it. **Divergence between the filing and the behaviour is the exposure** — the filing is a representation to a regulator, and the system is the fact.

### (C)(9) — Quarterly review of performance, use and outcomes

**Requires:** the performance, use and outcomes of the tool are reviewed and revised, if necessary and at least on a quarterly basis, to maximise accuracy and reliability.

**Evidence:** dated review records covering performance, use and outcomes; the revisions made or the reasoned decision not to revise; and the accuracy and reliability measures used.

**Test:** does the cadence survive a mid-quarter model version change on the provider's release schedule? If behaviour can change without a code change on your side, a quarterly review can describe a system that no longer exists. Record the model, prompt and tool versions in force for the period the review covers.

### (C)(10) — Purpose limitation on patient data

**Requires:** patient data is not used beyond its intended and stated purpose, consistent with HIPAA as applicable.

**Evidence:** the stated purpose for each patient-data flow into the tool; the enforcement point at which a use outside that purpose is refused; and evidence of refusal actually occurring.

**Test:** query the retrieval layer for patient data under a purpose it was not collected for. **Does the system refuse, or does it answer and rely on the requester's restraint?** A purpose boundary that exists only in a policy document is not an enforcement point. Retrieving broadly and filtering the response afterwards does not satisfy this — the data was disclosed into the context; only the display was suppressed.

### (C)(11) — No direct or indirect harm

**Requires:** the tool does not directly or indirectly cause harm to an enrollee.

**Evidence:** defined harm classes; the monitoring that would detect each; incident records and dispositions; and coverage of **indirect** harm — delay, added appeal burden, care abandonment — not only incorrect determinations.

**Test:** name the monitoring that would have caught a systematic delay introduced by the tool. Indirect harm rarely shows up in determination accuracy, which is usually the only thing being measured.

---

## Part C — The prohibition

**§ 15-10B-05.1(D):** *"An artificial intelligence, algorithm, or other software tool may not deny, delay or modify health care services."*

**Evidence:** an architectural demonstration that the tool has no write path to the determination of record — enforced technically, not by convention.

**Test:** attempt to make the tool issue an adverse determination without a provider decision. **If the only thing preventing it is process, the prohibition rests on process.** This is an entitlement question: what the automated actor is permitted to do, bounded at the point of action rather than described in a document.

---

## Part D — Reporting and production obligations

These sit outside § 15-10B-05.1 but bind the same systems, and each has an architectural precondition that is easy to miss until it is too late to fix retrospectively.

| # | Obligation | Source | The architectural precondition |
|---|---|---|---|
| D1 | Quarterly report to the Commissioner stating, for adverse decisions, whether a prior authorization or step therapy protocol was involved, the type of service at issue, and **whether an artificial intelligence, algorithm or other software tool was used in making the adverse decision** | § 15-10A-06, field added by Ch. 747 (2025) | The AI-involvement flag is **per adverse decision**. It must be captured at decision time. It cannot be reliably reconstructed later from logs, because "the tool was in the pipeline" is not the same claim as "the tool was used in making this decision." |
| D2 | Where adverse decisions for a service type grow **10% or more in the preceding calendar year, or 25% or more over three years**, report the changes in medical management contributing to the rise, other known reasons, and efforts made to determine the cause | § 15-10A-06(a)(2), as amended by Ch. 165 (2026) | An automatic trigger for a causal account. If a tool was deployed in that window, you will be asked whether it caused the rise — and you will need pre- and post-deployment comparability that was designed in, not assembled under examination. |
| D3 | On examination of a pattern of adverse decisions relating to emergency department services, the carrier **shall produce all documents related to an adverse decision, including documents or electronic documents in the possession of a private review agent acting on behalf of the carrier**; independent review may be ordered at the carrier's cost | § 15-10B-21, added by Ch. 165 (2026) | Production reaches **through the vendor**. A carrier's ability to comply depends on a third party's evidence model. Contractual audit rights are necessary and not sufficient — the records have to exist in a producible form. |

---

## Part E — Closure rule

An item closes only when the linked evidence **exists**, is **readable by an examiner**, **covers the specific determination selected**, and **survives the required retention period**.

"Designed," "planned," "normally," "the model is instructed to," and "the team knows" are not closure evidence.

Two further rules, specific to this regime:

- **Evidence produced under examination is not evidence.** Duties (C)(7) and (C)(8) create a standing state, not a response capability. If the artifact is assembled after the request, the obligation was not met during the period under review.
- **The filing is a representation.** Where the filed utilization plan under (C)(8) and the deployed system disagree, the divergence is the finding — regardless of which one is better.

---

## How to use this

1. Select **one material historical adverse determination** where a tool was involved. Not a synthetic case, and not the best-documented one.
2. Work Part B in order. Link the record that proves each duty was satisfied for that determination. Do not paste explanations into the evidence column.
3. Mark anything missing, indirect or unreconstructible as a gap, with an owner and a date.
4. Have the reconstruction performed by **someone outside the team that built the system**. The failure mode is familiarity, not competence.
5. Re-run Part B after any model, prompt, tool, criteria or data-source change — and at minimum on the quarterly cadence (C)(9) already requires.
6. If a vendor conducts any part of the review, run Parts A and D against **them**, not only against yourself.

---

*Sources: Maryland Insurance Article §§ 15-10A-06, 15-10B-05.1, 15-10B-21. Chapter 747 of 2025 (HB 820), approved 20 May 2025, effective 1 October 2025. Chapter 165 of 2026 (HB 1563), approved 28 April 2026, effective 1 June 2026 except as stated in that Act. Statutory text read from the enacted chapters published by the Maryland General Assembly. Citations verified 4 August 2026; check the primary source before relying on any of them.*