Identification of Predictors for Clinical Deterioration in Patients With COVID-19 via Electronic Nursing Records: Retrospective Observational Study

Introduction

As of September 27, 2023, the World Health Organization reported more than 770 million confirmed cases of COVID-19, including approximately 6.9 million deaths [1]. In South Korea, among 34,436,586 confirmed cases between January 3, 2020, and August 31, 2023, there were 35,812 deaths attributed to COVID-19 [2]. Among them, older patients or those with underlying diseases or comorbidities died due to severe conditions during the beginning of COVID-19; however, other cases exhibited initially mild symptoms that gradually worsened, causing death. Therefore, early detection of aggravating factors and symptoms is crucial for timely intervention and treatment.

COVID-19 symptoms include fever, cough, sore throat, nasal congestion, malaise, headache or muscle pain, dehydration, pneumonia, sepsis, and shortness of breath [3]. COVID-19 mortality prediction relies on patient age, blood oxygen saturation, and body temperature [4]. Through a systematic evaluation and external validation of 22 prognostic models for COVID-19 [5], admission oxygen saturation on room air and age have been shown to be strong predictors of clinical deterioration and mortality, respectively, in adults hospitalized with COVID-19.

The electronic health records (EHR) system contains detailed information about symptoms, problems, and services or care provided to patients with COVID-19. Previous studies have used EHR data to predict the COVID-19 prognosis [6-8]. However, nursing documentation records, which account for a substantial portion of EHR data, are underused for research owing to their low quality of documentation [9,10]. Unlike other EHR data, nursing documentation lacks standardization than other data such as diagnoses, laboratory, and medication data. The common data model (CDM) of the Observational Medical Outcome Partnership (OMOP) is designed to standardize the structure and content of observational data from multiple sites for efficient analyses that can produce reliable evidence; it is rarely used to standardize nursing documentation. One study [11] standardized 6277 nursing statements data using Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) and converted them into the OMOP CDM to develop a fall-prediction model along with other CDM data. SNOMED CT is the most comprehensive multilingual clinical terminology [12,13] used by more than 40 countries and 30,000 individuals or organizations. The SNOMED CT International Edition, published on April 30, 2023, includes 360,942 concepts, 1,595,980 descriptions (synonyms) of concepts, and 3,261,032 relationships between concepts. Despite these advancements, there is a lack of empirical studies analyzing patient outcomes using standardized nursing documentation or records with SNOMED CT.

The American Nurses Association in 2015 and 2018 position statements also recommended the use of SNOMED CT to code nursing assessments or outcomes to facilitate the interoperability of nursing data. Several studies have examined the content coverage of SNOMED CT in the nursing domain by mapping standardized nursing terminologies, such as the Nursing Intervention Classification, International Classification for Nursing Practice (ICNP), Clinical Care Classification System, North American Nursing Diagnosis Association-International, and Omaha system, into SNOMED CT [14-18]. Thoroddsen et al [19] described the nursing care of patients with COVID-19 using ICNP and SNOMED CT. SNOMED CT outperformed ICNP in representing COVID-19 diagnoses and interventions. In particular, SNOMED CT comprehensively covered nursing interventions for patients with COVID-19. However, prior research only evaluated its usability or content coverage in the nursing domain, with limited analysis of specific phenomena or problems. Therefore, analyzing specific phenomena or problems using standardized nursing documentation data with SNOMED CT is necessary.

This study aimed to (1) standardize the nursing documentation records of patients with COVID-19 using SNOMED CT and (2) identify predictive factors of clinical deterioration in patients with COVID-19 using standardized nursing records.

Methods

Study Design and Setting

This retrospective observational study used data extracted from a clinical data warehouse (CDW) at a tertiary hospital with 1782 beds in Seoul, South Korea. The hospital operated 2 general wards and 4 disaster intensive care units (DICU) to treat COVID-19.

Outcomes

In this study, the clinical outcome is the patient’s clinical deterioration. We defined clinical deterioration as an event in which patients with COVID-19 were transferred to the ICU from general wards.

Study Participants

In total, 460 patients with COVID-19 were admitted to 2 general wards for infectious disease treatment at the hospital between December 2019 and June 2022. Patients transferred from or to other general wards or ICUs other than the DICU were excluded. Among the 460 patients, 19 patients were transferred to the DICU because of their deteriorating clinical condition. The mean length of stay (LOS) in the general ward of the 441 patients who were not transferred to the DICU during hospitalization was 8.71 days. The mean and maximum LOS of the 19 patients transferred to the DICU were 3.11 and 7 days, respectively.

Considering LOS differences between the 2 groups, the final analysis of the study included 226 admitted patients with COVID-19 at 2 general wards who were discharged or transferred to the DICU within 7 days between December 2019 and June 2022. Among these, 207 patients who were discharged in a stable clinical condition from the hospital within 7 days were assigned to the control group. A total of 19 patients who were transferred to the DICU within 7 days because of their deteriorating clinical conditions were assigned to the case group.

Data Sources

Using CDW at a study hospital, the general characteristics of patients including age, gender, and initial respiratory symptoms were extracted from initial nursing assessment records. The patient’s acuity level and nursing statements were extracted from the nursing records. A total of 57,558 nursing statements from 226 patients were extracted. Among these, 45,852 nursing statements were obtained from the nursing records of 207 patients in the control group. Of the remaining records, 11,706 nursing statements were obtained from 19 patients in the case group.

Mapping to International Standard Terminology, SNOMED CT

Nursing statements were standardized by mapping them to SNOMED CT concepts. The 57,558 nursing statements consisted of 1776 unique nursing statements which were interface terminologies developed by Seoul National University Hospital. Among the 1776 unique nursing statements, 260 unique nursing statements which were recorded 50,867 in total (cumulative recorded rate of 90%) were selected as the mapping source. Among these 260 unique statements, the control group used 237 nursing statements, and the case group used 204 nursing statements.

According to the mapping guide in the previous study [20], 2 experts with more than 5 years of SNOMED CT mapping experience performed the mapping. The scope of the map was defined to restrict mapping to the precoordinated SNOMED CT concept only. Korean nursing statements were translated into English, and search terms were chosen considering the clinical context. SNOMED CT concepts are organized into 19 top-level concepts or hierarchies under the root concept. Among them, the Clinical finding (finding) hierarchy contains concepts related to symptoms and disorders. The procedure hierarchy contains concepts related to activities performed in the provision of health care and regime or therapy. The Situation with explicit context hierarchy includes concepts that specifically define the context information of a clinical finding or procedure [21]. Therefore, the nursing statements in nursing problems or outcomes and diagnosis domains were mapped to concepts of “clinical finding” and “situation with explicit context” hierarchies, and those in the nursing interventions domain were mapped to concepts of “procedure” hierarchy within the SNOMED CT International edition, released on November 30, 2022. If a SNOMED CT concept semantically consistent with the nursing statement could not be found, it was mapped to broader precoordinated concepts.

The mapping results were classified according to the level of correspondence as 3 types: “exact map,” “narrow to broad map,” and “no map.” Mappings were categorized as follows: “exact map” when the meaning of the nursing statement matched an equivalent SNOMED CT concept, “narrow to broad map” when the meaning of the nursing statement matched a broader SNOMED CT concept, and “no map” when there was no broad match concept for the meaning of the nursing statement.

Internal validation of the final map was conducted by calculating interrater reliability. Two experts reviewed the mapping results. If the mapper and reviewer selected the same result, the map was deemed correct. If the maps differed, the results were evaluated via a group discussion, and one of them was selected.

Statistical Analyses

The general characteristics of patients (eg, gender, age, and patient acuity level) in both the case and control groups were explored. For patients in the case group transferred to the DICU (average ward LOS, 3.1 days), nursing statements documented up to the third day were extracted. The total number of nursing statements was divided by the LOS to calculate the mean number of nursing statements per day. Thereafter, the data set was separated according to the top-level hierarchies of SNOMED CT mapped to nursing statements, “clinical finding” or “situation with explicit context,” and “procedure.” Among the data set, we excluded SNOMED CT concepts that do not describe the patient’s clinical problems, such as no breathlessness, free of symptoms, no sputum, and no cough. To identify key features of nursing statements associated with the exacerbation of patient condition, random forest algorithms were used, and optimal hyperparameters were selected with 2 data sets (“clinical finding” or “situation with explicit context” and “procedure”).

To assess the effects of the features identified by random forest algorithms on the exacerbation of patient conditions, we performed logistic regression. Patient DICU transfer status was the dependent variable, and the top 5 features from each hierarchy identified by random forest algorithms were independent variables at a significance level of α=.05. Considering the small sample size for binary logistic regression analysis, the model was fitted using a modified estimation procedure known as Firth correction [22]. R (version 4.2.2; R Foundation for Statistical Computing) was used for all analyses.

Ethical Considerations

This study was approved by the Institutional Review Board of the Seoul National University Hospital (H-2207-097-1341). The requirement for informed consent was waived according to the relevant guidelines and regulations of the institutional review board. Identifiers, such as the patient’s ID and name, were encrypted so that individuals could not be identified during the data analysis. Participant compensation was not offered since this study was a retrospective observational study.

Results

SNOMED CT Mapping Results

The results of mapping nursing statements to SNOMED CT are presented in Table 1. The interrater reliability of the mapping results was excellent (87.7%). Of the 260 unique nursing statements, 157 were on nursing problems or outcomes, 16 were on nursing diagnoses, and 87 were on nursing intervention statements. A total of 138 (87.8%) of nursing problems or outcomes statements were mapped to concepts within “clinical finding” and “situation with explicit context” hierarchies on SNOMED CT. Through a validation process, 19 nursing problems and outcome statements were mapped into concepts within the procedure hierarchy. All 16 nursing diagnosis statements were mapped to clinical finding concepts. Among the 87 nursing intervention statements, 85 (97.7%) nursing intervention statements were mapped to concepts within the procedure hierarchy. Only 2 statements were mapped to the clinical finding concepts, “267038008 |Edema|” and “29658002 |Oxygen supply absent|.” Of the 260 nursing statements, 244 (93.8%) nursing statements were classified as “exact map” and 16 (6.2%) nursing statements were classified as “broad map.” The mapped concepts with high frequency are presented in Multimedia Appendix 1.

Clinical Characteristics

The clinical characteristics of the study participants are presented in Table 2. The mean age was 55.9 (SD 17.3) years, and the mean LOS was 5.3 (SD 2.0) days. Patients in the control group were hospitalized in the ward for approximately 5.04 (SD 1.6) days. Patients in the case group were admitted to the ward for approximately 3.1 (SD 1.9) days. Overall, 119 (52.7%) patients were male. A total of 197 (86.7%) patients had a patient acuity level of 4, and 26 (11.5%) patients had a patient acuity level of 3. Additionally, 59 (26.1%) patients had a cough, 47 (20.8%) patients had sputum, and 26 (11.5%) patients had dyspnea as their initial symptoms.

Feature Selection via Random Forest

The feature importance calculated using the random forest method is shown in Figure 1. The most important feature in “clinical finding” showed “449171008 |Oxygen saturation below reference range (finding)|.” Prominent concepts that express respiratory issues included “267036007 |Dyspnea (finding)|,” “271823003 |Tachypnea (finding)|,” and “49727002 |Cough (finding)|.”

‎

Among procedure concepts, “57485005 |Oxygen therapy|” was the most important feature, followed by “284034009 |Pulse oximetry monitoring|,” “56342008 |Temperature taking (procedure)|,” “428426009 |Notification of physician|,” and “737612005 |Education about isolation for infection control|.”

Association Between Patient’s DICU Transfer and Selected Features

Table 3 presents the results of logistic regression analysis, which explored the association between patient DICU transfer status and features in both “clinical finding” or “situation with explicit context” and “procedure” selected via random forest. In “clinical finding,” patients with dyspnea and inadequate food diet were more likely to experience DICU transfer (dyspnea: odds ratio [OR] 5.99, 95% CI 2.25-20.29; inadequate food diet: OR 10.0, 95% CI 2.71-40.84). Patients with infectious disease in nursing records were less likely to be associated with the DICU transfer in patients with COVID-19 (OR 0.32, 95% CI 0.16-0.66). In the procedure, patients with “oxygen therapy” and “notification of physician” were more likely to experience DICU transfer (oxygen therapy: OR 1.89, 95% CI 1.25-3.05; notification of physician: OR 1.72, 95% CI 1.02-2.97). Patients with “temperature taking” were less likely to experience DICU transfer (OR 0.36, 95% CI 0.18-0.70).

Discussion

Principal Results

This study standardized the nursing statements of patients with COVID-19 in a hospital in South Korea by mapping them to SNOMED CT, an international standard terminology. Using these standardized statements, machine learning analysis with a random forest was conducted. Concepts that could predict patient deterioration in nursing problems or outcomes and intervention domains were identified. In the nursing problems or outcomes domain, concepts in nursing statements related to dyspnea, tachypnea, and oxygen saturation below the reference range were associated with patient deterioration. In nursing interventions, concepts in nursing statements related to respiratory intervention, including oxygen therapy, pulse oximetry monitoring, and physician notification, were associated with DICU transfer. These concepts were different from the findings of another study that analyzed nursing statements of patients with ovarian cancer. Kim et al [23] analyzed standardized nursing statements of patients who underwent curative surgery for epithelial ovarian cancer, and urination, food supply, bowel mobility, and pain were identified as the most common concepts.

To the best of our knowledge, this study is the first to identify predictive factors from nursing records standardized using SNOMED CT. Although previous studies [24-29] have used standardized nursing records such as the North American Nursing Diagnosis Association-International, Nursing Intervention Classification, Nursing Outcomes Classification, and ICNP to predict the patient’s condition or explore the nursing care provided, no study has predicted patient clinical deterioration using SNOMED CT–standardized nursing records. To achieve interoperable health information exchange, the Centers for Medicare and Medicaid Services and Office of the National Coordinator for Health Information Technology recommended the use of SNOMED CT and Logical Observation Identifier Names and Codes (LOINC) as reference terminologies. In 2020, the Office of the National Coordinator for Health Information Technology introduced the United States Core Data for Interoperability, a standardized set of health data classes and constituent data elements that recommended the use of international standard terminologies, including SNOMED CT and LOINC [30]. Consequently, standardization of nursing records using reference terminology before conducting data analysis is required in the nursing field.

Our study comprehensively standardized all nursing statements in study participants using SNOMED CT concepts. Several previous studies have evaluated SNOMED CT coverage in various health care domains, such as frailty and wound care [31-34]. In nursing, efforts have been made to map ICNP 7-axis concepts, nursing problem lists, and ICU nursing flowsheets to SNOMED CT [16,32,35-37], which provides the most detailed semantic expression compared with other standard terminologies [19]. Notably, this study unexpectedly mapped 19 nursing statements on nursing assessments or outcomes to the “procedure” hierarchy, despite the lack of corresponding concepts in the “clinical finding” or “situation with explicit context” hierarchies. This was due to the fact that the problem or phenomenon was the result of specific nursing procedures or practices. For example, the statement “a patient is applying a mattress to prevent bedsores” in nursing assessment or outcome was mapped to concept “733920005 |Provision of pressure-relieving mattress|” in the procedure hierarchy.

The identified features affecting patient deterioration (DICU transfer) in the hierarchy of “clinical finding” included respiratory issues such as low oxygen saturation, dyspnea, and tachypnea. Izquierdo et al [7] reported patient signs and symptoms, especially tachypnea, to be reliable predictors of DICU admission, which is consistent with our results. In addition, inadequate diet was significantly associated with DICU transfer, potentially due to the inability of patients to eat independently owing to their deteriorated condition. These findings highlighted the importance of signs and symptoms in nursing records for predicting a patient’s deteriorating clinical condition. Alternatively, nursing statements related to “infectious disease” within “clinical finding” and “temperature taking” within “procedure” decreased the risk of clinical deterioration in patients with COVID-19. Since patients with COVID-19 were included as study participants, it was valid that the concept of “infectious disease” appeared frequently in nursing statements, but repeated studies are needed to explain the finding that the more statements related to this concept, the lower the risk of clinical deterioration. Considering previous studies [4,38] reporting “fever” as a significant factor affecting deterioration in patients with COVID-19, our study revealed that increased nursing activity of “temperature taking” notably reduced patient deterioration risk. This suggested that nurses may have preemptively frequently monitored temperature to prevent fever; however, to accurately interpret this result, analyzing “body temperature” data is necessary. The features identified in this study highlight the potential of nursing records as a valuable real time predictor of a patient’s clinical condition.

Limitations

This study had some limitations. First, the analyzed data were extracted from a single tertiary university hospital, which limits the generalizability of this study. Second, sample bias is possible because patients admitted for more than 7 days to the ward were excluded. Third, other clinical findings or nursing interventions may not have been documented as nursing statements in the nursing records. Considering that this study extracted only structured nursing statements stored in the CDW, records written in free text were not included in the analysis. Fourth, as SNOMED CT concepts were used in the analysis, the results may vary depending on the hierarchical structure or level of the mapped SNOMED CT concept. For example, concepts such as “chest pain,” “headache,” and “pain” derived through random forest are connected in a hierarchical structure where “chest pain” and “headache” are subcategories of the “pain” concept. If “chest pain” and “headache” were grouped together as “pain,” the effect of “pain” on clinical deterioration might have been significant.

Conclusions

This study showed that standardized nursing records are an important source of data that can be used to predict clinical deterioration in patients with COVID-19. In total, 260 nursing statements were mapped to the SNOMED CT, including 109 concepts in the clinical finding hierarchy, 73 concepts in the procedure hierarchy, and 22 concepts in “situation” with an explicit context hierarchy. Among the standardized nursing statements, key clinical findings were respiratory issues, including low oxygen saturation, dyspnea, and tachypnea. The primary procedure-related features included oxygen therapy, pulse oximetry, and temperature monitoring. In specific, low oxygen saturation, dyspnea, tachypnea, and oxygen therapy are associated with the risk of clinical deterioration in patients with COVID-19. This study validates the use of nursing records as variables for predicting the deterioration of patients with COVID-19. Future research should investigate the integration of standardized nursing records with diagnoses, laboratory, and medication data to develop a highly reliable predictive model.