Predicting COVID-19 prognosis in the ICU remained challenging: external validation in a multinational regional cohort

Meijs, Daniek AM; van Kuijk, Sander MJ; Wynants, Laure; Stessel, Bjorn; Mehagnoul-Schipper, Jannet; Hana, Anisa; Scheeren, Clarissa IE; Bergmans, Dennis CJJ; Bickenbach, Johannes; Vander Laenen, Margot; Smits, Luc JM; van der Horst, Iwan CC; Marx, Gernot; Mesotten, Dieter; van Bussel, Bas CT

doi:10.1016/j.jclinepi.2022.10.015

Predicting COVID-19 prognosis in the ICU remained challenging: external validation in a multinational regional cohort

Author:

Meijs, Daniek AM

van Kuijk, Sander MJ ; Wynants, Laure ; Stessel, Bjorn ; Mehagnoul-Schipper, Jannet ; Hana, Anisa ; Scheeren, Clarissa IE ; Bergmans, Dennis CJJ ; Bickenbach, Johannes ; Vander Laenen, Margot ; Smits, Luc JM ; van der Horst, Iwan CC ; Marx, Gernot ; Mesotten, Dieter ; van Bussel, Bas CT

Keywords:

Science & Technology, Life Sciences & Biomedicine, Health Care Sciences & Services, Public, Environmental & Occupational Health, COVID-19, SARS-CoV-2, Critical care, Intensive care unit, Prediction, Prognosis, INDIVIDUAL PROGNOSIS, MISSING DATA, SEVERITY, MODEL, EXPLANATION, Humans, Female, Middle Aged, Aged, Male, Cohort Studies, Intensive Care Units, Critical Care, Hospital Mortality, Retrospective Studies, CoDaP Investigators, Intensive Care Unit, 01 Mathematical Sciences, 11 Medical and Health Sciences, Epidemiology, 4202 Epidemiology

Abstract:

OBJECTIVES: Many prediction models for coronavirus disease 2019 (COVID-19) have been developed. External validation is mandatory before implementation in the intensive care unit (ICU). We selected and validated prognostic models in the Euregio Intensive Care COVID (EICC) cohort. STUDY DESIGN AND SETTING: In this multinational cohort study, routine data from COVID-19 patients admitted to ICUs within the Euregio Meuse-Rhine were collected from March to August 2020. COVID-19 models were selected based on model type, predictors, outcomes, and reporting. Furthermore, general ICU scores were assessed. Discrimination was assessed by area under the receiver operating characteristic curves (AUCs) and calibration by calibration-in-the-large and calibration plots. A random-effects meta-analysis was used to pool results. RESULTS: 551 patients were admitted. Mean age was 65.4 ± 11.2 years, 29% were female, and ICU mortality was 36%. Nine out of 238 published models were externally validated. Pooled AUCs were between 0.53 and 0.70 and calibration-in-the-large between -9% and 6%. Calibration plots showed generally poor but, for the 4C Mortality score and Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC) score, moderate calibration. CONCLUSION: Of the nine prognostic models that were externally validated in the EICC cohort, only two showed reasonable discrimination and moderate calibration. For future pandemics, better models based on routine data are needed to support admission decision-making.