Publication Spotlight
Unlike traditional AI models trained for one specific task using datasets tediously labeled by experts, foundation models are models that learn from massive, multimodal, unlabeled datasets and can be adapted to a wide variety of tasks. While their adaptability opens up possibilities in radiology, there are also challenges ahead.
Following publication of their paper “Foundation Models in Radiology: What, How, Why, and Why Not?” in Radiology, Magda Paschali, PhD, Postdoctoral Scholar in the Stanford Radiology AI Development and Evaluation (AIDE) Lab, and Akshay Chaudhari, PhD, Assistant Professor in the Department of Radiology, and of Biomedical Data Sciences, and co-director of the AIDE Lab discussed some of these key considerations with Linda Chu, MD, Associate Editor of the RSNA podcast.
#1
By learning a variety of different tasks together, the overall performance across all tasks can also increase
Figure from Paschali et al
“By having a foundation model learn the differences between, say, a breast arterial calcification on a mammogram and any suspicious lesion and by introducing the model to a variety of the different tasks, we can see that the performance across all tasks improves. This is the rationale behind using a lot of these large scale models..”
– Akshay
#2
We need to pair the building of foundation models with high-quality evaluation
“Foundation models are trained on very large data sets, but might not reach the high accuracy that one could get with a task specific model that was specialized on a particular application.”
– Magda
Figure from Paschali et al
“If I have a task of segmenting organs on a CT scan or trying to detect lesions on mammogram, what is actually better: a model that is task specific for that one singular task or a more general purpose foundation model?”
– Akshay
#3
Task-specific AI models aren’t going away
Figure from Schneider et al showing an example of how foundation models and task-specific AI models can be incorporated together into the AI ecosystem.
“I think the future is moving towards broader systems where multiple models work in conjunction with one another. While this setting requires some task specific models and some foundation models, but key is understanding the end to end data flow and what clinical task the data supports.”
– Akshay
“[Foundation models] won’t necessarily be a replacement [for task-specific AI models], but they will create a shift so we can integrate both approaches and research which model is better suited for which task.”
– Magda
Maier-Hein, L., Reinke, A., Godau, P. et al. Metrics reloaded: recommendations for image analysis validation. Nat Methods 21, 195–212 (2024). https://doi.org/10.1038/s41592-023-02151-z
Lekadir K, Frangi A F, Porras A R, Glocker B, Cintas C, Langlotz C P et al. FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare BMJ 2025; 388 :e081554 doi:10.1136/bmj-2024-081554
