Why Every Healthcare AI Recommendation Needs a Receipt

Buy a coffee and you get a receipt. What you bought, what it cost, who sold it to you, when. Nobody calls that a technological achievement. It is simply the minimum a transaction produces so the people involved can account for it later.

Healthcare AI mostly does not produce one.

A model flags a patient. A number lands in a worklist. Somebody acts on it, or does not. Thirty days later an outcome appears in a report, and nothing connects any of those moments to each other. We are holding a clinical decision to a lower standard of documentation than a four-dollar coffee.

What a receipt would actually contain

Six fields. None of them exotic.

1. What was detected.

The specific signals, not the composite score. A refill fourteen days overdue, an authorization expiring, no follow up since discharge.

2. Why it mattered.

The reasoning that turned those signals into a flag, in language a clinician can disagree with.

3. What was recommended.

The actual suggested action, not a risk tier.

4. Who owned it.

A name, not a department.

5. What was done.

Whether the call happened, whether anyone picked up, whether the appointment was booked. Attempted and completed are different events.

6. What happened next.

The outcome, attached to the chain rather than floating in a quarterly report.

Read that list again and notice something. Not one of those fields requires a better model. They are all record-keeping.

Why almost nothing produces them today

The reason is structural rather than negligent. The model lives in an analytics platform. The worklist lives in the EHR. The outreach lives in a care management tool, or a spreadsheet, or somebody's memory of a Tuesday afternoon. The outcome arrives months later in claims.

Four systems, four owners, and no field anywhere that holds the thread.

So the organization ends up with two piles of data that cannot be joined. One pile says which patients the model worried about. The other says which patients did badly. The thing nobody can produce is the middle, which is the only part anyone could have changed.

This is why good models fail quietly

Here is the consequence, and it is more expensive than it looks.

When a program underperforms, the post mortem almost always lands on the model. Accuracy gets questioned, a new vendor gets evaluated, somebody proposes a pilot. But without a receipt, nobody in that room can distinguish between two completely different failures. Either the model flagged the wrong patients, or it flagged the right ones and nothing happened.

Those require opposite fixes. One is a data science problem. The other is a staffing and workflow problem, and no amount of model retraining will touch it.

A systematic review of twenty three studies found clinicians overriding decision support alerts at rates between 46 and 96 percent. Read that as a model quality finding and you will go buy a better model. Read it as a receipt problem and you start asking which alerts were overridden, by whom, and what happened to those patients afterward. Only the second question has an answer that changes anything.

The regulation already did half of this

This is where the argument stops being philosophical.

The ONC HTI-1 final rule, published in January 2024, requires certified health IT that includes predictive decision support to disclose thirty one source attributes across nine categories. Development details and input features. Fairness process. External validation. Quantitative performance measures. Ongoing maintenance and revalidation schedules. Since January 1, 2025, that criterion sits inside the Base EHR definition, which makes it a condition of certification rather than a best practice.

It is a genuinely serious rule, and it is a receipt. For the model.

Every one of those nine categories describes how the thing was built. Not one of them describes what it did to a patient on a Tuesday. The industry now has a well regulated answer to "is this model sound" and no answer at all to "what did it recommend for a patient, call her Mrs. Alvarez, who decided what to do about it, and did it help."

CMS gestured at the same gap from a different direction. Its February 2024 guidance to Medicare Advantage plans confirmed that algorithms may inform coverage decisions, but that a denial has to rest on the individual's circumstances rather than on broader data patterns alone. That is a receipt requirement in everything but name. You cannot demonstrate that an individual's circumstances were considered unless something recorded which ones.

The direction of travel is obvious. Transparency about the model came first because it was easier to legislate. Accountability for the recommendation is what comes next, and organizations that wait for the rule will be building it under deadline instead of on purpose.

Three readers, three uses

A receipt is not one artifact serving one audience.

A clinician can see the reasoning and argue with it, which is the entire difference between a tool they trust and one they click past. A quality team can audit what actually happened rather than what was supposed to happen, which is the work that quality professionals have been doing by hand, with chart review, for as long as the discipline has existed. And finance can finally answer the question every program eventually gets asked, which is whether the thing worked.

The receipt is the measurement layer

The objection to all of this is that it sounds like documentation burden, and nobody in healthcare has appetite for more of that.

I would argue the opposite. The receipt is not overhead sitting beside the work. It is the only mechanism by which a program can prove it did anything at all. Strip it out, and you are left running an expensive intervention on faith, hoping the aggregate numbers drift the right way and having no account to give when they do not.

That standard is what we are building toward at Preventra, and I would rather the whole category moved there than that it became anyone's differentiator. A model that cannot show its work is a liability dressed as an asset, and the quality nurses who have been building these receipts by hand, one chart at a time, deserve better than that.

A recommendation that cannot produce a receipt is a suggestion, not decision support.