← News

Models & toolsSources: arXiv · 19 Nov 2026

Blinded audit tests whether an LLM can probe black-box clinical models

Researchers planted flaws in copies of stroke prediction models and had GPT-6 Astra find them through prediction queries alone.

In plain words

A clinical prediction model estimates risk from patient data, but its inner workings are often hidden. Researchers trained models on stroke trial data, secretly inserted one flaw into each of 24 copies, and let a large language model interrogate them by asking for predictions only. The model found nearly every planted flaw, though many of its extra warnings needed verification.

What was announced

The team, writing on arXiv, trained four model types on the International Stroke Trial and planted one flaw in each of 24 copies, mixing them with 8 clean models. GPT-6 Astra audited each model three times, blind, through a tool returning only predictions, with judging rules fixed in advance. Its surrogate models reproduced the audited models' log-odds with a mean R² of 0.976, and it located the planted flaw in 69 of 72 runs (96%). A purpose-written script found all 24 flaws.

The audit produced many additional warnings: 60% of 1,089 checkable reports failed a pre-specified check against the model's average behaviour, though post hoc analysis suggested most reflected real but small steps, and 15% of checkable reports did not correspond to anything in the model. Of 601 unexpected relationships the LLM read from clean models, 11% held in later patients, rising to 33% after a training-data check. The authors conclude that reports must be checked against the model and discoveries against data.

What it does not show

  • The evidence is the paper's abstract only; full methods, thresholds and caveats in the complete article are not covered here.
  • Results come from one dataset (International Stroke Trial), one LLM and one audit design; performance on other clinical models or settings is not established.
  • The 96% detection figure carries a wide confidence interval (80–99%) reported by the authors.
  • The LLM's extra warnings and discoveries required additional checks; the abstract does not establish they are reliable without verification.

For security and IT teams

Scope
The audit relied solely on query access to model predictions, not on inspecting model internals.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.