Using Predictive Models to Improve Outcomes Following Surgery and Critical Illness

The OR and ICU are two of the most data-rich environments in all of health care. This wealth of information provides the substrate for identifying patients who are at risk for postoperative complications and implementing targeted, risk-mitigating interventions to those patients. Although the anesthesiology community has taken early steps toward achieving this goal, there is still a long way to go. I am an anesthesiologist and intensivist at Washington University in St. Louis who has been fortunate to receive a Mentored Research Training Grant from the Foundation for Anesthesia Education and Research (FAER) to help launch my career in examining how we can use innovative tools such as predictive analytics to improve outcomes following surgery and critical illness.

Naturally, the first step in this process is to train and validate algorithms that predict postoperative complications. This requires datasets with rich information on patient phenotypes and outcomes, ideally with granular details about vital signs, medications, and other interventions. Such datasets are readily available for ICU patients, but they did not exist for surgical patients at the time my team’s work began (Sci Data 2016;3:160035; Sci Data 2018;5:180178). Therefore, our multidisciplinary group assembled a dataset containing minute-by-minute details for more than 110,000 surgical patients from our institution, and the dataset has continued to grow ever since. Using these data, we have built deep neural networks, gradient boosted models, and other cutting-edge machine learning algorithms that predict postoperative mortality, postoperative acute kidney injury, and other complications with excellent discrimination (Br J Anaesth 2019;123:688-95; Br J Anaesth 2020;125:230-31; AMIA Annu Symp Proc 2020;2019:343-52; JAMA Netw Open 2021;4:e212240). My computer science colleagues continue to explore new ways to improve these models.

Although high-quality predictive models are necessary to deliver targeted interventions to high-risk patients, models alone are not sufficient – they must be incorporated into clinical workflows. Little is known about how anesthesiology clinicians want to interact with predictive models, despite the growing number of models being reported in our field’s literature. Therefore, I led a study using interviews, simulated patient evaluations, and think-aloud techniques to understand when and how anesthesiology clinicians want the results of predictive models to be presented to them (Anesth Analg June 2023). The results indicated that the electronic health record (EHR) is the ideal location for this information, that distinguishing between modifiable and nonmodifiable risk is important, and that the user interface needs to be simple, with effective use of color and graphs. Based on these findings, we constructed an EHR-integrated graphical user interface that achieved high levels of satisfaction from users.

With good predictive models and an acceptable user interface, the next question is whether using models changes the way that clinicians assess their patients. We recently completed a randomized trial of 5,071 patients that addressed this question (F1000Res 2022;11:653). We asked anesthesiology professionals with varying backgrounds and levels of experience – ranging from attending anesthesiologists to resident physicians to nurse anesthetists – to conduct thorough evaluations of patients at the start of surgery and to predict how likely the patient was to die within 30 days or experience acute kidney injury within seven days. In half of the cases, the clinicians used our predictive models in addition to the EHR and any other data sources they typically use, while clinicians in the other half did not use our predictive models. This trial will yield exciting insights regarding how the models impacted the clinician predictions, including changes in outcome discrimination between the groups.

As predictive models continue to mature and become integrated into clinical workflows, clinicians need more guidance about what actions to take when a patient is identified as having elevated risk. Are changes to the management plan needed, such as different blood pressure goals or fluid management strategies? Could enhanced postoperative monitoring help prevent failure-to-rescue events that lead to unplanned ICU admission, ICU or hospital readmission, or death? If enhanced monitoring is desired, how do we scale such interventions in a world where there are ongoing shortages of both physicians and nurses? Thoughtfully constructed telemedicine programs can help clinicians provide this sort of support simultaneously for patients in multiple geographic locations, increasing the feasibility of scaling without requiring extra clinicians in every location. Working with our institution’s tele-critical care program, we recently launched a pilot study of tele-critical care consultation for hospitalized patients at high risk for unplanned ICU admission (Appl Clin Inform 2024;In Press). In this pilot, tele-critical care physicians and nurses are using two-way audio-video communication technology to comanage these at-risk patients along with the bedside care team for a predetermined period of time, with the goal of providing earlier access to critical care services when they are needed and preventing some patients from requiring ICU care.

Like all areas of clinical care and research, the work of creating and implementing predictive models requires an effective team. My longtime partner-in-investigation, Dr. Christopher King, is an incredibly talented anesthesiologist, biostatistician, and data scientist. We have been privileged to work with collaborators from Washington University’s Department of Anesthesiology, Department of Surgery, Department of Computer Science & Engineering, and Institute for Informatics, in addition to clinical staff at Barnes-Jewish Hospital. I am also thankful for a wonderful group of mentors, funding from FAER, and a supportive anesthesiology department. With a team like this, I am excited for what the future holds!