Inspired by how doctors often seek second opinions, EMM is used in parallel to the primary AI model and contains an ensemble of five independent AI submodels that perform the same diagnostic task. These submodels process the same input image in parallel to the primary AI model and compare the results. If the majority of the submodels agree with the primary AI model’s prediction, then clinicians can have high confidence in the diagnosis. If there is little agreement, that signals a need for more scrutiny.

Crucially, EMM doesn’t need access to the inner workings of the commercial AI system it monitors, making it compatible with black-box systems already in use across hospitals.

Why It Matters

The study authors tested the EMM on over 2,900 CT scans, using it to quality-check both an FDA-cleared commercial AI model and an open-source model. They found that EMM reliably flagged cases where there was uncertainty in the primary model’s AI predictions – especially for images that contained subtle or ambiguous findings – while confirming high-confidence predictions for more obvious cases.

In more than half the cases, EMM increased the clinicians’ confidence in the AI’s output. In others, it helped flag cases that warranted extra attention. Importantly, EMM only failed to detect that the primary AI model output was incorrect in a small minority (4%) of cases.

EMM doesn’t just assess confidence. It also suggests an action – a next step. Using confidence thresholds, EMM can categorize confidence in the primary AI model’s predictions into increased, similar, or decreased confidence categories, helping clinicians know when to trust the AI and when to use their own judgment.

The value of these suggested next steps was also evaluated by looking at the cognitive burden of reviewing “false alarms”, or predictions from the primary AI model that EMM flagged as “low confidence” but were actually correct. In this benefit-to-drawbacks analysis, EMM delivered substantial accuracy gains by drawing radiologist attention to cases that truly needed a second look. For images containing an intracranial hemorrhage, EMM increased relative accuracy by up to 38% while maintaining low false-alarm rates under 1%.

For images without intracranial hemorrhage, the benefits were more nuanced, with net gains at an intracranial hemorrhage prevalence rate typically seen in emergency departments. In contrast, at lower prevalence levels typically seen in inpatient or outpatient settings, the marginal accuracy improvements were outweighed by the burden of unnecessary reviews. These results highlight EMM’s effectiveness in high-stakes scenarios while also underscoring the importance of tailoring the confidence thresholds set for next steps to specific disease prevalence rates and use cases.