Sniffing out Sepsis - Vibes vs Scoring Systems?
/Knack SKS, Scott N, Driver BE, et al. Early Physician Gestalt Versus Usual Screening Tools for the Prediction of Sepsis in Critically Ill Emergency Patients. Ann Emerg Med 2024
Background
Sepsis remains an increasingly common emergency department condition that is tied to higher morbidity and mortality across the United States as well as the rest of the world. National campaigns to improve sepsis care, namely the Centers for Medicare and Medicaid Services’ introduction of “Severe Sepsis and Septic Shock Early Management Bundle”, have faced many challenges in physician and provider adherence. Sepsis as a disease process has been difficult to both clearly define and quickly recognize. Many metrics for recognition and management of sepsis are dependent upon various scoring systems, including SIRS, SOFA, qSOFA, and MEWS, none of which were designed for the acute detection of sepsis within the emergency department. Many electronic medical records have also begun implementing AI-generated scoring systems that flag physicians that a patient may be septic, though these systems have not been well studied. This prospective observational study aimed to compare the sensitivity and specificity of emergency physician gestalt for early detection of sepsis with that of various scoring models as well as a LASSO-generated computer model.
Population
Patients were adult (18 or older) patients at a single academic emergency department who were triaged to a resuscitation bay and were not classified as a trauma, STEMI, stroke, cardiac arrest, or active labor.
Methods
After an appropriate patient was triaged into the resuscitation bay, researchers would approach the attending emergency physician and ask him or her to use a visual analog scale (VAS) to answer the question “What is the likelihood that this patient has sepsis?”, with 0 being no infection/highly unlikely and 100 being infection/very likely. The physician was asked to do this at time point 15 minutes from patient arrival and 60 minutes from patient arrival. Meanwhile, the researchers collected real-time data on vital signs as well as what data was or was not available in the computer system at 15 minutes and 60 minutes. Then, they inputted this data into SIRS, SOFA, qSOFA, and MEWS scoring systems to determine if sepsis was considered likely from each system. The gold standard comparison was a final discharge diagnosis of sepsis that was marked as present at time of admission. They also developed a machine learning model using Least Absolute Shrinkage and Selection Operator (LASSO) using 80% of the available data. They then tested the LASSO model using the remaining 20% of data to see how well it could detect sepsis at the same time points.
Results
Physician gestalt outperformed every single scoring system, including the LASSO system, in both specificity and sensitivity, at both 15 minutes (82.5% sensitivity, 84.9% specificity) and 60 minutes (85.1% sensitivity, 87.2% specificity). There was no significant difference in results between the two time points. Of the scoring systems, the machine learning LASSO model was superior.
Limitations
This study has several limitations. The scoring systems, LASSO model included, function more in a binary system rather than a percentage likelihood of sepsis. However, attendings were asked to choose a percentage likelihood that a patient was septic rather than a binary “yes” or “no”, which may conflate findings and alter how the statistical analysis was performed. Importantly, only patients triaged to resuscitation bays were enrolled. It is likely that most emergency physicians find the early detection of sepsis to be more challenging in a patient whose illness is less apparent rather than in a patient who is clearly sick enough to be triaged directly to the highest acuity zone of a department. Additionally, patients who are brought directly to a resuscitation bay will have some data and assessment at 15 and 60 minutes, but patients who are not may spend a significant period of time in the waiting room where far less robust data is available. As mentioned earlier, the scoring systems physicians are compared to were never designed to be implemented in such a way, though this is a reasonable comparison given they are now often utilized in sepsis bundle metrics.
Authorship
Written by: Megan Wright, PGY-3, University of Cincinnati Department of Emergency Medicine
Editing and Posting by Jeffery Hill, MD MEd, Associate Professor, University of Cincinnati Department of Emergency Medicine
Cite As: Wright, M., Hill, J. Sniffing out Sepsis - Vibes vs Scoring Systems? www.tamingthesru.com. www.tamingthesru.com/blog/journal-club/sepsis-vibes. 7/24/24
