The Tuesday, December 3, 2024, issue of the RSNA Daily Bulletin highlighted the AIDE lab’s work in training open-source, large language models (LLMs) to assess the completeness of clinical histories that accompany imaging orders. 

The study results were presented by Arogya Koirala, machine learning engineer, during an oral presentation. In his talk, Koirala explained how the AIDE lab automated the previously tedious task of evaluating if a clinical history contained complete and relevant information.  

The trained model found that 26% of the analyzed imaging orders accompanying MRI, CT, ultrasound, and X-ray orders from an adult and pediatric emergency department of a tertiary academic medical center were considered complete – meaning that the clinical history contained all five elements of past medical history, what, where, when, and clinical concern. When weighted for the clinical history elements considered most important, the completeness rate increased to 74%, which may be partly due to a dedicated quality improvement project at the study site to improve the completeness of clinical histories. 

This benchmark completeness rate enables comparisons to the completeness rate at other sites, with the automated approach potentially helping others take on similar quality improvement projects. Another surprising advantage of the study approach was that although both ChatGPT and open-source AI models were trained, the smaller open-source model performed at the same level as the frontier ChatGPT model, which has significant implications for compute resources and clinical data privacy concerns. 

Read more here: https://dailybulletin.rsna.org/2024/tue/tue07