Overcrowded intensive care units. Lack of personal protective equipment. A scramble for hospital beds and ventilators. Healthcare workers marginalized. The COVID-19 pandemic has exposed many well-documented vulnerabilities in healthcare systems. The need for accurate and early clinical assessment of COVID-19 severity has been critical to the development of crisis care standards to deal with the growing pandemic. These standards of care are based on mortality prediction models that assess the risk of impending death in patients.
Shortly after the first case of COVID-19 was officially diagnosed in Colorado, UCHealth University of Colorado Hospital (UCH) reached out to a team led by Tell Bennett, MD, MS, associate professor of biomedical informatics at the CU School of Medicine to create a mortality prediction model specifically for COVID-19 patients.
“COVID forced us to crystallize our thinking around mortality prediction models,” recalls Bennett. “The pieces were out there. People thought about crisis standards of care, but they never materialized. We never tested whether existing predictors would meet our needs. They have not.”
Bennett has already focused on the use of informatics and data science in healthcare, including supporting clinical decision-making in intensive care units. His experience as an attending physician in the pediatric intensive care unit at Children’s Hospital Colorado and as the informatics director at the Colorado Clinical and Translational Sciences Institute laid the foundation for a COVID-19 mortality prediction model.
Bennett worked against the clock with IT staff from UCHealth, data scientists from CU Anschutz Medical Campus, frontline clinicians and ethicists to develop a tool that extracts data from patients’ Electronic Health Records (EHRs) in real-time.
“We deployed it in the UCHealth EHR,” says Bennett, who is also a co-leader of the National COVID Cohort Collaborative (N3C) Data Enclave, which collects COVID-19 data from institutions across the country. “It’s running in the background. We applied a relatively innovative modeling strategy to make it accurate and interpretable.”
An introduction to mortality prediction models
Prior to COVID-19, most crisis standards for treatment protocols relied on sequential organ failure assessment (SOFA), a scoring system that assesses multiple organ systems (neurologic systems, blood, liver, kidney, and blood pressure) and assigns a score based on the data received in each category. The higher the SOFA score, the higher the risk of death. However, the SOFA score proved inadequate during the 2009 swine flu pandemic, particularly in patients with respiratory failure.
Bennett’s team added four other predictive models to SOFA, including a pneumonia score and the Charlson Comorbidity Index, which measures the risk of death for hospitalized patients with multiple co-occurring conditions (such as diabetes and heart failure). Then they integrated COVID-specific predictors: lactate dehydrogenase (LDH; sometimes associated with viral infection) and ferritin (an indicator of inflammation). Within six weeks, they launched the predictor and began collecting data across UCHealth’s 12 hospitals.
“This was early in the pandemic and clinicians were working incredibly hard in very difficult conditions,” says Bennett. “Our team of data scientists felt they built this tool with purpose because it was a way to contribute at a very uncertain time.”
Over the next 10 weeks, the model collected data from thousands of hospitalized patients with and without COVID-19. Bennett’s team compared these values to those of patients hospitalized before the pandemic and found that the new model was more accurate than any other individual model in predicting death from COVID-19.
Although the model is constantly collecting data and updating the score every 15 minutes, Bennett cautions that the tool should not be used in isolation to make treatment decisions for individual patients.
The goal of the new model has always been to provide information for multidisciplinary decision-making teams in large healthcare systems who may be responsible for the difficult task of triaging patients in a crisis scenario. As COVID-19 evolves and new variants emerge, adjustments to the mortality predictor need to be made.
Although no crisis care standards had to be used in Colorado and the mortality prediction model was never used for its intended purpose, Bennett says creating the tool was an invaluable experience that demonstrates what can be accomplished with data from an EHR.
“We continue to work with UCHealth and discuss a more comprehensive program to build and deploy models to meet their operational needs,” said Bennett. “The success has shown what we can achieve together.”
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of the press releases published on EurekAlert! by contributing institutions or for the use of information about the EurekAlert system.