Efficacy of AI models in Detecting Clinical Deterioration
Clinical question: Can an artificial intelligence (AI) enabled intervention model reduce the risk of clinical deterioration and subsequent care escalation in hospitalized patients?
Background: Clinical deterioration in hospitalized patients poses a significant risk of morbidity and mortality but identification by clinicians may be delayed. Automated early warning systems have been developed to help clinicians recognize these patients. Epic Deterioration Index (EDI)—an ordinal logistic regression model that predicts the risk of composite outcome of Rapid Response Team (RRT) activation, intensive care unit (ICU) transfer, cardiopulmonary arrest, or death—is the most frequently used of these systems. However, its efficacy remains unproven.
Study design: Retrospective cohort study using regression discontinuity design (RDD)
Setting: Stanford University Hospital
Synopsis: A total of 9,938 patients were included from January 2021 to November 2022. EDI scores were calculated every 15 minutes from 31 clinical measures captured by the electronic health record. A score of 65 (range 0 to 100) was used as the threshold for high risk of clinical deterioration, which triggered an alert sent to the patient’s nurse and physician to initiate collaborative measures, including a structured huddle and checklist.
The primary outcome was escalation in care, including RRT activation, ICU transfer, or cardiopulmonary arrest. The secondary outcome was a composite of RRT, ICU transfer, cardiopulmonary arrest, and death. RDD analysis showed an absolute risk reduction of 10.4% (95% confidence interval [CI], –20.1 to –0.8; P=0.03) in the primary outcome.
Limitations: While the study showed a significant reduction in escalation of care, there was no reduction in mortality. The study did not assess the impact of the AI-enabled intervention model in isolation, but rather as a step in a cascade of interventions that included a collaborative workflow between the nurse and physician. Generalizability may be limited since the study was conducted at a single academic hospital.
Bottom line: An AI-enabled deterioration model can significantly reduce the risk of escalation in care for hospitalized patients. This study provides much-needed evidence for continued development and study of such early warning systems.
Citation: Gallo RJ, et al. Effectiveness of an Artificial Intelligence-Enabled Intervention for Detecting Clinical Deterioration. JAMA Intern Med. 2024;184(5):557-62.
Dr. Hakimzada
Dr. Hakimzada is a hospitalist in the division of hospital medicine at the Mount Sinai Health System and an assistant professor of medicine at the Icahn School of Medicine at Mount Sinai in New York.