When a medical AI system harms a patient, who is responsible?

The honest answer right now is that nobody fully knows. The law is unsettled. But the direction the scholars thinking hardest about this are leaning should get every hospital leader's attention — because it is not leaning toward the vendor.

It is leaning toward you.

Let me be precise, because this is where the brochure version of AI risk collapses into something real. The question has three possible targets: the clinician who used the tool, the hospital that deployed it, and the developer who built it. Traditional rules fit none of them cleanly. Algorithms are opaque. Their failures do not trace the way a defective device does. And "who was in control" is genuinely unclear for a system designed, trained, and updated by a vendor — but selected, integrated, and signed off by a hospital.

The emerging argument — from the legal scholars thinking hardest about this — is enterprise liability: the hospital is the party best positioned to be responsible. The reason is uncomfortable, and it will not go away. The hospital selects the tool, integrates it into the workflow, supervises the humans who use it, and makes the clinical decision that touches the patient. Those are the levers of control — and in the law, control is where accountability lands. Hospitals already carry vicarious liability for the errors of the practitioners they credential, and evolving doctrine treats negligent selection and retention of an AI system as their own exposure. A model does not assume the hospital's obligations.

Now the caveat — and it is the caveat that saves you.

Liability should shift to the developer only when the developer gates the information the hospital needs to govern the tool: the training data, the parameters, the version history, the local performance. If a vendor will not let you interrogate the model — will not let you see what drives its recommendations — then the vendor, not you, should carry the risk of what you cannot see.

Which means your contract is not procurement. It is liability allocation.

A vendor that locks its thresholds and hides its calibration data is not selling you a tool. It is asking you to hold a liability you have no way to defend. The black-box defense is not an engineering position. It is a refusal to let you be responsible — which is the same thing as refusing to let you be sovereign.

That is why the contract is where the pause lives. Before deployment, demand the rights that make responsibility possible: the right to pause the tool when its performance degrades, the right to audit the vendor's model, the right to inspect what you are being asked to stand behind. Without those rights, you have accepted the risk and refused the information.

Accountability cannot be outsourced. It can be shared — the clinician owns the clinical decision in context, the vendor and the institution own the integrity of the system — but it cannot be handed off. The patient's question is not "which box in the org chart." It is "who remains responsible when this is used?" If your institution cannot answer that in one sentence, that is not a legal gap. It is a governance one.

That is not governance. That is a liability with a signature on it.

Where does your institution stand? Take the free AI Governance Readiness Audit — a sixty-second diagnostic on whether you have governance, or just a charter and a meeting cadence.