Clinical AI audit finds safe final diagnoses often hide risky decision paths
A new arXiv paper proposes auditing each step an AI system takes during diagnosis, reporting that many hospital cases reached correct answers through decisions rated potentially harmful.
In plain words
Researchers argue that judging a clinical AI system only by its final diagnosis misses risks in the steps taken along the way. They built a method that replays hospital cases checkpoint by checkpoint and rates every decision for potential patient harm. In their tests, many cases with correct endings still passed through unsafe decisions, and fixing one error sometimes created new ones.
What was announced
The authors say most evaluations score final diagnoses, and even stepwise benchmarks that release evidence over time check only whether systems reach the right answer or gather the right information, not whether the decisions a patient experiences are safe. Their trajectory audit replays real admissions, lets each system decide using only evidence released so far, and rates every test, treatment and disposition for potential harm and clinical validity against the documented final diagnosis and physician-written rubrics. An LLM judge scores each decision, calibrated against a physician reference standard built from triple annotation and senior adjudication of a stratified sample.
Across 300 cases drawn from a hospital cardiology cohort, MIMIC-IV and the public ASCENT benchmark, with up to seven AI systems per source, the paper reports that 55.2 percent of hospital trajectories reaching an acceptable final diagnosis passed through a decision with moderate or severe potential harm, and 24.8 percent through a clinically unacceptable one. Harm rates were 76.2 percent on MIMIC-IV and 39.6 percent on ASCENT. In an oversight audit branching at each first actionable error, structured corrections resolved 73.6 percent of first errors versus 24.8 percent after bare rejection, but new potentially unsafe decisions followed 30.2 percent of corrections versus 19.7 percent of rejections. The authors conclude that a correct final diagnosis does not certify a safe trajectory, and a successful correction does not certify a safe recovery.
What it does not show
- The evidence is the paper's abstract only; the full article's methods, case selection details and caveats could not be checked.
- The reported figures come from the authors' own evaluation and have not been independently confirmed.
- The abstract does not identify the AI systems tested or state whether the approach has been deployed in practice.
For security and IT teams
- Scope
- The evaluation covers 300 cases from a hospital cardiology cohort, MIMIC-IV and the public ASCENT benchmark, with up to seven AI systems per source, so findings may not extend to other specialties or settings.
- Status
- The work appears as a new arXiv abstract dated 18 November 2026 and describes a proposed audit method rather than a deployed system.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.