An innovative artificial intelligence tool aimed at assisting healthcare professionals in determining the risk of sepsis among hospital patients showcases a unique characteristic: it acknowledges its own uncertainty and recommends which demographic data, vital signs, and lab test results could enhance its predictive capability.
This system, named SepsisLab, was created with insights from medical practitioners working in emergency rooms and intensive care units (ICUs), where the life-threatening condition of sepsis frequently occurs. Clinicians expressed dissatisfaction with a current AI-supported application that produces risk scores solely based on electronic health records, without input from healthcare providers.
Researchers at The Ohio State University designed SepsisLab to assess a patient’s risk of sepsis within four hours. As this timeframe progresses, the system identifies any lacking patient information, evaluates its importance, and visually depicts for clinicians how specific details will influence the final risk assessment. Trials that utilized both public and private patient data indicated that incorporating 8% of the suggested information led to an 11% increase in sepsis prediction accuracy.
“The conventional model follows a typical human-AI competition format, causing frequent false alarms in ICUs and emergency departments without considering clinician input,” stated Ping Zhang, the senior author of the study and an associate professor in computer science and biomedical informatics at Ohio State.
“Our approach seeks to integrate AI into every step of decision-making by implementing the ‘AI-in-the-human-loop’ strategy. We’re not merely constructing a tool; we’ve also involved physicians in the project. This represents a true partnership between computer scientists and medical professionals to create a system centered around human input that empowers doctors.”
The findings were presented in the publication KDD ’24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining on August 24 and will be discussed further during an oral presentation on August 28 at SIGKDD 2024 in Barcelona, Spain.
Sepsis is a critical medical condition that can quickly lead to organ failure, making it challenging to diagnose due to its symptoms—such as fever, low blood pressure, elevated heart rate, and breathing difficulties—often resembling those of other illnesses. This research builds on previous machine learning efforts by Zhang and his team that aimed to identify the optimal timing for administering antibiotics to patients suspected of having sepsis.
SepsisLab is engineered to quickly generate a risk assessment and to refresh this prediction every hour as new patient information becomes available.
“When a patient is first admitted, there are often many missing data points, particularly in lab test results,” explained Changchang Yin, the lead author and a PhD student in computer science and engineering at Zhang’s Artificial Intelligence in Medicine lab.
In conventional AI models, missing data is typically filled in with a single estimated value through a technique called imputation. “However, imputation can introduce uncertainty that may impact subsequent predictions,” Yin noted.
“If the imputation model struggles to accurately estimate a crucial missing value, that variable should ideally be acquired. Our active sensing algorithm aims to identify such gaps and inform clinicians about which additional variables may be necessary to observe—elements that can enhance the prediction accuracy.”
It is equally vital to minimize uncertainty over time and to provide clinicians with practical recommendations. These include lab tests ranked by their importance in the diagnostic process and estimates of how a patient’s sepsis risk might change based on specific medical interventions.
Experimental results demonstrated that integrating 8% of this additional data from lab tests, vital signs, and other high-value variables reduced the level of uncertainty in the model by 70%, which was instrumental in achieving an 11% boost in sepsis risk prediction accuracy.
“The algorithm highlights the most crucial variables, and clinician actions help to further decrease uncertainty,” Zhang remarked, also a key faculty member at Ohio State’s Translational Data Analytics Institute. “This underlying mathematical framework is the primary technical innovation—the foundation of our research.”
Zhang envisions human-centered AI as a significant part of future medicine, but emphasizes that such systems must build trust with clinicians.
“This isn’t about constructing an AI system to dominate the world,” he clarified. “The essence of medicine revolves around hypothesis testing and making nuanced decisions continuously, rather than simply answering ‘yes’ or ‘no.’ We foresee a scenario where a human drives the interaction, utilizing AI to enhance their capabilities.”
This research received support from the National Science Foundation, the National Institutes of Health, and an Ohio State President’s Research Excellence Accelerator Grant. Furthermore, Zhang has obtained additional NIH funding to sustain collaborative efforts with clinicians in this area.
Other co-authors include Jeffrey Caterino from The Ohio State University Wexner Medical Center, alongside Bingsheng Yao and Dakuo Wang from Northeastern University, and Pin-Yu Chen from IBM Research.
