A Machine Learning–Based Prediction Model for Acute Kidney Injury in Patients With Community-Acquired Pneumonia: Multicenter Validation Study

Introduction

Acute kidney injury (AKI) is common in patients with community-acquired pneumonia (CAP), which is associated with higher morbidity and mortality, longer hospital stays, and increased financial costs. In recent years, outbreaks of pneumonia caused by new pathogens, including SARS in 2002, Middle East respiratory syndrome in 2012, and the COVID-19 pandemic in 2019, have severely strained resources and had significant societal impacts. In addition, a considerable percentage of patients with CAP, ranging from 4.3% (429/10,069) to 34.4% (631/1836), develop AKI [Chawla L, Amdur R, Faselis C, Li P, Kimmel P, Palant C. Impact of acute kidney injury in patients hospitalized with pneumonia. Crit Care Med. 2017;45(4):600-606. [CrossRef] [Medline]1-Srisawat N, Murugan R, Lee M, Kong L, Carter M, Angus DC, et al. GeneticInflammatory Markers of Sepsis (GenIMS) Study Investigators. Plasma neutrophil gelatinase-associated lipocalin predicts recovery from acute kidney injury following community-acquired pneumonia. Kidney Int. Sep 2011;80(5):545-552. [FREE Full text] [CrossRef] [Medline]3]. Identifying and intervening in AKI early can facilitate the timely restoration of renal function, ultimately enhancing survival rates and aiding patients in their recovery post discharge [Fiorentino M, Tohme FA, Wang S, Murugan R, Angus DC, Kellum JA. Long-term survival in patients with septic acute kidney injury is strongly influenced by renal recovery. PLoS One. 2018;13(6):e0198269. [FREE Full text] [CrossRef] [Medline]4,Vijayan A, Abdel-Rahman EM, Liu KD, Goldstein SL, Agarwal A, Okusa MD, et al. Recovery after critical illness and acute kidney injury. Clin J Am Soc Nephrol. 2021;16(10):1601-1609. [FREE Full text] [CrossRef] [Medline]5]. However, AKI is primarily detected through elevated levels of serum creatinine (Scr), which is a delayed indicator of AKI. Although various novel urine and plasma biomarkers have shown promising results in diagnosing AKI, only a few are presently used in clinical practice [Alge JL, Arthur JM. Biomarkers of AKI: a review of mechanistic relevance and potential therapeutic implications. Clin J Am Soc Nephrol. Jan 07, 2015;10(1):147-155. [FREE Full text] [CrossRef] [Medline]6]. Furthermore, the cost associated with biomarker assessment and the challenges in interpreting the results have limited their widespread adoption.

In recent years, machine learning (ML) techniques have offered the potential to develop accurate prediction models for AKI, enabling early risk stratification and personalized interventions. A retrospective study published in Nature in 2019 included a total of 703,782 adult patients from 172 hospitals and 1062 outpatient facilities. By using the recurrent neural network algorithm, this study demonstrated the ability to predict 55.8% (18,837/33,759) of patients with AKI 48 hours in advance, with an impressive area under the receiver operating characteristic curve (AUC) of 92% [Tomašev N, Glorot X, Rae JW, Zielinski M, Askham H, Saraiva A, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature. 2019;572(7767):116-119. [FREE Full text] [CrossRef] [Medline]7]. These findings suggest that ML can accurately predict the occurrence of AKI at an early stage.

Numerous studies have focused on ML models for predicting AKI in patients. However, these models have primarily focused on vigorous patients, patients with sepsis, patients with cardiac surgery, patients with cancer, and other specific populations. Currently, there is a lack of models for predicting AKI in patients with CAP. Furthermore, most of the studies have relied on databases like MIMIC-III (The Medical Information Mart for Intensive Care III), with a scarcity of local patient databases and a lack of external validation. Therefore, there is an urgent need to develop AKI prediction models specifically tailored to the population of patients with CAP to provide targeted guidance for diagnosis and treatment. This study aimed to use ML methods to develop models that accurately predict CAP-associated AKI (CAP-AKI).

Methods

Study Population

This retrospective cohort study involved 3 independent tertiary hospitals—Wuxi People’s Hospital, Nanjing First Hospital, and Sir Run Run Hospital affiliated with Nanjing Medical University. Data were collected from January 2016 to December 2017. The inclusion criteria included patients aged 18 years and older who were diagnosed with CAP during hospitalization. The study exclusion criteria were (1) patients diagnosed with stage 4-5 chronic kidney disease (CKD) or receiving regular renal replacement therapy, (2) patients with a history of kidney transplant, (3) patients with a hospital stay of less than 24 hours or exceeding 90 days, (4) patients who had been treated by other medical institutions within 14 days before hospitalization, (5) patients who relied on mechanical ventilation and palliative care for advanced tumors on a chronic basis, (6) occurrence of AKI within 24 hours of admission or 7 days thereafter, and (7) patients with peak Scr levels less than 53 μmol/L or Scr measurements fewer than 2 times between 1 year before hospitalization and discharge or fewer than 1 time during hospitalization.

Primary Outcome

The primary outcome for developing the ML models was AKI in patients with CAP. AKI is identified based on the Scr level, as stated by the Kidney Disease Improving Global Outcomes (KDIGO) [Kellum JA, Lameire N, KDIGO AKI Guideline Work Group. Diagnosis, evaluation, and management of acute kidney injury: a KDIGO summary (Part 1). Crit Care. 2013;17(1):204. [FREE Full text] [CrossRef] [Medline]8]. It is defined as an increase in Scr to ≥1.5 times baseline within 7 days of admission, or an increase in Scr ≥0.3 mg/dL within 48 hours. The baseline Scr level was determined as the average within 7 to 365 days before hospitalization. If baseline Scr is not recorded, the first Scr value upon admission is used as the baseline Scr. Urine output was not considered for AKI diagnosis due to missing data [Yang L, Xing G, Wang L, Wu Y, Li S, Xu G, et al. Acute kidney injury in China: a cross-sectional survey. Lancet. Oct 10, 2015;386(10002):1465-1471. [CrossRef] [Medline]9]. Estimated glomerular filtration rate (eGFR) was calculated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) creatinine equation [Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF, Feldman HI, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. May 05, 2009;150(9):604-612. [FREE Full text] [CrossRef] [Medline]10]. Proteinuria is defined as “The presence of at least 1+ protein in urinalysis.”

Training and Validation Data

The dataset was divided into 2 parts: one from Wuxi People’s Hospital for training and internal validation (2568/3211, 80% for model development and 643/3211, 20% for validation) and the other from the multicenter patients for external validation.

Data Processing

Baseline characteristics and clinical data of patients within 24 hours of admission, including 62 indicators, were obtained from the electronic record system. Variables with over 15% (9/62) missing values were excluded, and the remaining variables were imputed using the missForest method. Outliers were identified and treated as missing values. The data of each feature were standardized using the z score method. Variables with strong collinearity (variance inflation factor [VIF] value≥10) were discarded using the VIF. The synthetic minority over-sampling technique (SMOTE) was applied to address data imbalance and improve the prediction efficiency of the models.

Feature Selection and Machine Learning Models Establishment

Feature importance was evaluated using the sklearn library method. Features were sorted in descending order based on their scores. A sliding windows sequential forward feature selection technique method based on random forest (RF) and out-of-bag (OOB) filtering was used to select features related to AKI. The optimal number of features is determined by finding the subset that results in the lowest OOB error (or the highest OOB score). After feature selection, 5 ML algorithms, including logistic regression (LR), RF, support vector machine, extreme gradient boosting (XGBoost), and deep forest (DF) models, were used to construct the prediction models. A detailed description of ML methods was given in

Multimedia Appendix 1

Detailed description of the machine learning methods.

Model Validation and Performance

External datasets were used to validate the models, and their performances were evaluated using various metrics such as the AUC, specificity, sensitivity, positive predictive value (PPV), negative predictive value, F₁-score, expected calibration error (ECE), calibration curve, and decision curve. A 1000-bootstrap method was used to determine CIs by sampling with replacement on the prediction indices.

Explanation of the Model

Shapley additive explanations (SHAP) is a method that constructs an additive interpretation model. It considers all features as “contributors” and calculates the marginal contribution of each feature when added to the model output. SHAP aims to provide explanations for “black box models” at both global and local levels, making the model more understandable and applicable.

On the other hand, local interpretable model-agnostic explanations (LIME) modify a single data sample by adjusting the eigenvalues and observing its effect on the output. LIME acts as an “interpreter” by explaining the prediction of each data sample. In LIME, a set of interpretations represents the contribution of each feature to the prediction of a single sample.

Statistical Analysis

To compare baseline characteristics across different datasets, we used 1-way ANOVA and chi-square tests. Statistical significance was determined using a 2-sided P value of less than .05. The ML models were created using Python (version 3.9.7; Python Software Foundation) with packages such as scikit-learn, NumPy, SHAP, and pandas, as well as R (version 4.1.0; R Core Team) with the glmnet package.

Ethical Considerations

The study was approved by the institutional review boards of Wuxi People’s Hospital (KY22073), Sir Run Run Hospital (2021-SR-042), and Nanjing First Hospital (KY20230410-01-KS-01). The study was conducted in accordance with the declaration of Helsinki. Due to its retrospective design, the ethics committees from Wuxi People’s Hospital, Sir Run Run Hospital, and Nanjing First Hospital waived the need for informed consent. Data can be analyzed secondary without additional consent. The personal information of patients was kept confidential throughout the entire study. Since the experimental design did not involve any intervention for the patients, no compensation was paid to the patients in the study.

Results

Baseline Characteristics

The study comprised a total of 6371 patients, with 3211 (50.4%) patients assigned to the training and internal validation groups, and 3160 (49.6%) patients assigned to the external validation group (Figure 1). Out of the total patients, 2566 (40.2%) were female, with a mean age of 67.0 (SD 17.6) years and a baseline Scr level of 72.5 (SD 31.1) μmol/L. Table 1 illustrates the differences between the training and validation sets.

‎

Among the 6371 patients, 1006 (15.8%) developed AKI, with the median time for AKI diagnosis being day 3 (IQR 2-4) post admission. A comparison between patients with and without AKI is presented in Table 2. Patients with AKI exhibited higher age, temperature, heart rate, breathing rate, and lower diastolic blood pressure levels. They also had a higher incidence of hypertension, CKD, cardiovascular disease, cerebrovascular diseases, acute respiratory failure, and acute cardiac dysfunction and a higher rate of mechanical ventilation compared with those without AKI. Furthermore, patients with CAP having AKI demonstrated significantly elevated baseline Scr, white blood cell count, neutrophil count, C-reactive protein (CRP), and blood urea nitrogen (BUN) levels compared with those without AKI (all P<.05). Conversely, baseline eGFR, platelet count, hemoglobin, and blood albumin levels were substantially lower in the non-AKI group (all P<.05).

Model Validation and Performance

In the training model, 9 variables with over 15% (9/62) missing values were excluded from the final cohort. These variables included blood lactic acid, ferritin, glycosylated hemoglobin C, blood oxygen partial pressure, blood carbon dioxide partial pressure, sputum pathogens, complement C3, complement C4, and procalcitonin. Among the remaining 53 variables, 3 had VIF values≥10. These variables were inhaled oxygen concentration, oxygen inhalation status, and vasoactive drug application. After eliminating variables with collinearity, the remaining 50 features (including age, sex, smoking status, and severe pneumonia), comorbidities (including history of chronic lung disease, CKD, malignancy, hypertension, diabetes, cardiovascular disease, chronic liver disease, and cerebrovascular disease), complications (such as septic shock, acute respiratory failure, and acute cardiac dysfunction), vital signs (including temperature, heart rate, respiratory rate, systolic blood pressure, and diastolic blood pressure), treatment on the day of admission (including noninvasive ventilation, mechanical ventilation, antiplatelet therapy, anticoagulation, renin-angiotensin-aldosterone system [RAAS] inhibitors, proton pump inhibitors, statins, diuretics, nonsteroidal anti-inflammatory drugs, insulin, calcium channel blockers, beta-blockers, and corticosteroids), and laboratory tests (including baseline Scr, baseline eGFR, CRP, urine protein, serum albumin, white blood cell count, platelet count, hemoglobin, neutrophil count, monocyte count, lymphocyte count, BUN, alanine aminotransferase [ALT], aspartate aminotransferase [AST], serum potassium, serum chloride, and serum sodium) were evaluated using the sliding windows sequential forward feature selection technique method.

The model generated an OOB value for each included feature. Figure 2 indicates that as the number of features increased to 11, the OOB value (0.579773) decreased substantially, and subsequently, the OOB value remained stable with increasing features. When the number of features reached 40, the OOB value (0.0340017) reached its lowest point. Therefore, to facilitate clinical application, we selected the first 11 features as the final model variables. These variables included sex, temperature, breathing rate, diastolic blood pressure, CRP, albumin, white blood cell count, platelet count, hemoglobin, neutrophil count, and BUN. Figure 3 illustrates that the BUN and neutrophil count consistently ranked among the top 5 indicators in all 5 models, suggesting their significant contribution to the occurrence of AKI.

‎

‎

Internal Validation of the Model

A total of 643 patients were included in the internal validation, with an AKI incidence of 13.5% (n=87). Figure 4A illustrates the AUC of the 5 models in the internal validation set. Among them, the DF model exhibited the highest discrimination compared with the other 4 models, achieving an AUC of 0.89 (95% CI 0.84-0.92). The XGBoost model followed closely with an AUC of 0.88 (95% CI 0.84-0.92). Detailed prediction performance of the 5 ML models is presented in Table 3. The DF model demonstrated the best specificity prediction for AKI (0.99, 95% CI 0.98-0.99). Compared with the other models except DF, the XGBoost model showed higher accuracy (0.92, 95% CI 0.90-0.94), PPV (0.78, 95% CI 0.68-0.86), and F₁-score (0.63, 95% CI 0.54-0.70). Calibration curves of the 5 models are depicted in Figure 4B. The results indicated that the DF model exhibited the lowest ECE value of 0.03, signifying superior calibration. The XGBoost model followed with an ECE of 0.05. Decision curve analysis of the prediction models is presented in Figure 4C. The findings revealed that the DF, XGBoost, and RF models yielded net benefits within the threshold range of 0-0.8. In the internal validation set, the DF model demonstrated the highest predictive ability for AKI in patients with CAP.

‎

External Validation of the Model

The data for the external validation set were obtained from Nanjing First Hospital and Sir Run Run Hospital affiliated with Nanjing Medical University, encompassing 3160 patients with CAP. The incidence of AKI was 20.4% (n=645) and the baseline clinical features are outlined in Table 1. Figure 5A presents the AUC of the 5 models in the external validation set. Among them, the DF model exhibited the most favorable predictive performance, achieving an AUC of 0.87 (95% CI 0.85-0.88), specificity of 0.93 (95% CI 0.93-0.94), and PPV of 0.63 (95% CI 0.59-0.67). The XGBoost model closely followed with an AUC of 0.86 (95% CI 0.85-0.88), accuracy of 0.84 (95% CI 0.83-0.85), and F₁-score of 0.65 (95% CI 0.63-0.68). Detailed performance metrics are presented in Table 4. Calibration curves of the prediction models are depicted in Figure 5B. The results demonstrated that the DF model exhibited the best calibration ability in the external validation set. Figure 5C presents the decision curve analysis curves of the 5 models, indicating that the DF, XGBoost, and RF models yielded net benefits within the threshold range of 0-0.6. In the external validation set, the DF model showcased the highest predictive ability for AKI in patients with CAP.

‎

SHAP Values Evaluate Feature Importance

We used SHAP to determine the importance ranking of the 11 features, as shown in Figure 6A. The significance of SHAP features is depicted in Figure 6B, where the x-axis represents the positive or negative contribution of a single feature to the model. The y-axis displays the 11 features in descending order of importance based on the absolute value of SHAP. A higher absolute value of SHAP indicates a greater contribution of the feature to AKI risk. The results revealed that BUN, neutrophil count, CRP, white blood cell count, breathing rate, and temperature made positive contributions, implying that higher values increased the risk of AKI. Conversely, female sex, albumin, platelet count, diastolic blood pressure, and hemoglobin made negative contributions, indicating that higher values decreased the risk of AKI.

‎

In addition to explaining the overall contribution of the model using SHAP, we used LIME to individualize the contribution of the last 11 features in the prediction process. Figure 7A illustrates the predictive process for a patient without AKI. First, the 11 patient characteristics were ranked by importance. Blue indicates the contribution to the prediction as non-AKI, orange represents the contribution as AKI, and the corresponding prediction probability is displayed on the left. The specific values of the patient’s 11 features are presented on the right. In this case, the patient had a BUN level of 5.0 mmol/L, neutrophil count of 9.52×10⁹/L, breathing rate of 20 breaths/min, temperature of 36.8°C, white blood cell count of 10.6×10⁹/L, albumin level of 24.6 g/L, CRP level of 0.5 mg/L, diastolic blood pressure level of 85 mm Hg, hemoglobin level of 110 g/L, and platelet count of 161×10⁹/L. The probability of predicting non-AKI for this patient was 1.0.

‎

Furthermore, Figure 7B demonstrates the predictive process for a patient with AKI. With a BUN level of 19.1 mmol/L, neutrophil count of 11.5×10⁹/L, breathing rate of 30 breaths/min, temperature of 37.8°C, white blood cell count of 12.4×10⁹/L, albumin level of 34.2 g/L, CRP level of 1 mg/L, diastolic blood pressure level of 80 mm Hg, hemoglobin level of 137 g/L, and platelet count of 240×10⁹/L, the probability of predicting AKI for this patient was 0.9.

Establishment of a Web-Based Prediction Platform

We developed a web-based prediction platform specifically designed for predicting AKI in patients with CAP (Figure 8). This platform uses the 11 selected variables as inputs to generate a risk value for predicting CAP-AKI. If the predicted risk value exceeds the predefined threshold of 74.5% set by the system, the result is considered positive, indicating that the patient falls into the high-risk category for developing CAP-AKI.

‎

Discussion

Principal Findings

In this study, we used ML algorithms to develop an innovative prediction tool for CAP-AKI. CAP stands as a prevalent infectious disease globally, entailing considerable morbidity and mortality [Rothberg MB. Community-Acquired Pneumonia. Ann Intern Med. Apr 2022;175(4):ITC49-ITC64. [CrossRef] [Medline]11-Peghin M, Bouza E. Community-acquired pneumonia: is less more? Lancet Infect Dis. 2022;22(2):159-161. [CrossRef] [Medline]13]. Among the array of complications linked with CAP, AKI has surfaced as a notable concern [Hoşgün D, Aydemir S. Factors affecting 90-day mortality in community and hospital acquired pneumonia patients with or without acute kidney injury. Afr Health Sci. 2022;22(3):567-577. [FREE Full text] [CrossRef] [Medline]14,Latief M, Ahmed Para R, Shafi O, Hassan Z, Farooq S, Abbas F. Incidence and risk factors of acute kidney injury in patients hospitalized with pneumonia: a prospective observational study. Med J Islam Repub Iran. 2021;35:150. [FREE Full text] [CrossRef] [Medline]15]. Our study unveils a noteworthy incidence of 15.8% for AKI among patients with CAP, indicating a substantial incidence of AKI within this population. Pinpointing the risk factors associated with AKI and crafting a predictive model are pivotal endeavors, enabling effective risk stratification and timely intervention for AKI. In this study, we identified the top 11 important predictors, including BUN, neutrophil count, respiratory rate, CRP, temperature, albumin, diastolic blood pressure, hemoglobin, platelet count, white blood cell count, and sex, as these variables can be easily collected in medical practice. In addition, our DF model showed good performance in both internal (AUC 0.89, 95% CI 0.84-0.92) and external (AUC 0.87, 95% CI 0.85-0.88) validation sets, indicating strong generalization ability.

ML techniques offer a promising path by harnessing the extensive pool of clinical data to offer insightful early-stage predictions of AKI. This approach potentially eliminates the necessity for additional AKI biomarker testing [Dong J, Feng T, Thapa-Chhetry B, Cho B, Shum T, Inwald D, et al. Machine learning model for early prediction of acute kidney injury (AKI) in pediatric critical care. Crit Care. 2021;25(1):288. [FREE Full text] [CrossRef] [Medline]16,Yue S, Li S, Huang X, Liu J, Hou X, Zhao Y, et al. Machine learning for the prediction of acute kidney injury in patients with sepsis. J Transl Med. 2022;20(1):215. [FREE Full text] [CrossRef] [Medline]17]. Hsu et al [Hsu C, Liu C, Tain Y, Kuo C, Lin Y. Machine learning model for risk prediction of community-acquired acute kidney injury hospitalization from electronic health records: development and validation study. J Med Internet Res. 2020;22(8):e16903. [FREE Full text] [CrossRef] [Medline]18] constructed an AKI prediction model using 10 preprocedural indicators through an XGBoost algorithm, revealing an AUC of 0.767 on derivation and 0.761 on validation for any stage of AKI. Other researchers also found that XGBoost [Luo X, Kang Y, Duan S, Yan P, Song G, Zhang N, et al. Machine learning-based prediction of acute kidney injury following pediatric cardiac surgery: model development and validation study. J Med Internet Res. 2023;25:e41142. [FREE Full text] [CrossRef] [Medline]19] and recurrent neural network [Kim K, Yang H, Yi J, Son H, Ryu J, Kim Y, et al. Real-time clinical decision support based on recurrent neural networks for in-hospital acute kidney injury: external validation and model interpretation. J Med Internet Res. 2021;23(4):e24120. [FREE Full text] [CrossRef] [Medline]20] could perform well in predicting cardiac surgery–associated AKI and in-hospital AKI. Due to the distinct etiology of AKI in patients with CAP compared with other causes such as cardiac surgery–related AKI or sepsis-induced AKI, there are differences in the characteristics and factors required for model establishment. Some unique features need to be incorporated, including respiratory status, body temperature, blood pressure, and inflammatory markers, which have a significant impact on the occurrence of AKI. These parameters may differ from those in AKI models related to other etiologies. This aspect has been further elaborated in the discussion section.

In this study, we used ML algorithms to develop an innovative prediction tool for CAP-AKI. Furthermore, we determined the top 11 important predictors in the DF model as prediction model variables as these variables can be collected easily in medical activities. These features comprehensively consider the patient’s basic characteristics (such as age and sex), disease status (such as blood pressure and Scr), and potential causes (such as infections and chronic diseases), providing a well-rounded informational foundation for the model from multiple perspectives. Similarly, AUC for AKI in patients with CAP of the internal validation and external validation cohorts was shown to be 0.86 (95% CI 0.84-0.92) and 0.87 (95% CI 0.85-0.88), respectively. In addition, we constructed SHAP to provide personalized interpretation for each patient. Subsequently, an online web risk calculator model for AKI in patients with CAP was established to predict the occurrence of AKI within 7 days upon patients’ hospital admission. The median time for diagnosing AKI in our study was the third day. Although there was a certain time difference between the diagnosis time of AKI and the time of the indicators we included, our results showed that the model still exhibited good predictive performance in the validation set, and we believed that this time difference could be used as the best time for early intervention in high-risk patients who may develop AKI.

Among the 5 ML models used in this study, LR is commonly used in clinical settings. However, its performance is suboptimal when dealing with large-sample and high-dimensional data. RF exhibits good generalization ability and can prevent overfitting. XGBoost derived from RF is unaffected by multicollinearity and possesses characteristics such as flexibility and efficiency. DF is a novel ensemble-based method proposed in recent years, serving as an alternative to deep neural networks. It combines various ensemble-based methods with indivisible modules. Compared with deep neural network, DF has the advantage of achieving the optimal model with a small amount of training data, without the need for backpropagation, and has low computational costs. Zhang et al [Zhang H, Wang Z, Tang Y, Chen X, You D, Wu Y, et al. Prediction of acute kidney injury after cardiac surgery: model development using a Chinese electronic health record dataset. J Transl Med. 2022;20(1):166. [FREE Full text] [CrossRef] [Medline]21] found that the DF model achieved an AUC of 0.881 for predicting AKI after cardiac surgery, outperforming other ML models, including LR, XGBoost, and RF. Since its proposal, DF has been widely applied and has demonstrated good predictive performance in various fields such as radiological diagnosis of COVID-19 and emotion recognition [Sun L, Mo Z, Yan F, Xia L, Shan F, Ding Z, et al. Adaptive feature selection guided deep forest for COVID-19 classification with chest CT. IEEE J Biomed Health Inform. 2020;24(10):2798-2805. [FREE Full text] [CrossRef] [Medline]22,Cheng J, Chen M, Li C, Liu Y, Song R, Liu A, et al. Emotion recognition from multi-channel EEG via deep forest. IEEE J Biomed Health Inform. 2021;25(2):453-464. [CrossRef] [Medline]23].

In this study, we conducted a ranking analysis of the included feature variables and observed that several factors, namely BUN, neutrophil count, respiratory rate, CRP, temperature, albumin, diastolic blood pressure, hemoglobin, platelet count, white blood cell count, and sex, exerted a significant impact on the final model. Among these features, BUN emerged as the most crucial indicator for AKI. Similar to Scr, BUN serves as a reflection of renal function and carries substantial predictive weight for AKI occurrence. These findings align with the diagnostic criteria recommended by the KDIGO and are consistent with previous clinical investigations. Elevated levels of white blood cells, neutrophils, and CRP are also risk factors for AKI in patients. AKI is closely associated with renal and systemic inflammation. In the presence of infection, inflammatory factors play a critical role in combating microbial pathogens and facilitating tissue repair. White blood cells, including neutrophils, can infiltrate the injured kidneys through the circulatory system, triggering the production of cytokines, chemokines, and other inflammatory mediators, ultimately leading to renal damage [Akcay A, Nguyen Q, Edelstein C. Mediators of inflammation in acute kidney injury. Mediators Inflamm. 2009;2009:137072. [FREE Full text] [CrossRef] [Medline]24]. CRP, as a plasma protein synthesized in the liver, serves as a nonspecific marker for systemic inflammatory response during the acute phase and is closely related to the severity of patients with CAP. Consistent with the findings of Wang et al [Wang W, Xie Y, Huang X, Zhou Y, Luo L. The value of N-terminal pro-brain natriuretic peptide and hs-CRP in predicting acute kidney injury after acute myocardial infarction. Am J Transl Res. 2022;14(8):5501-5510. [FREE Full text] [Medline]25] in patients with AKI after myocardial infarction, we also found that elevated CRP levels were a risk factor for AKI in the cohort of patients with CAP. Hypoalbuminemia is another risk factor for AKI in patients with CAP. Animal experiments have shown that albumin can counteract the decline in arterial vascular reactivity caused by endotoxemia, alleviate ischemia-reperfusion injury, and possess anti-inflammatory effects. Therefore, reduced albumin levels significantly reduce a patient’s resistance to infection and stress damage [Artigas A, Wernerman J, Arroyo V, Vincent J, Levy M. Role of albumin in diseases associated with severe systemic inflammation: pathophysiologic and clinical evidence in sepsis and in decompensated cirrhosis. J Crit Care. 2016;33:62-70. [CrossRef] [Medline]26]. Similar conclusions have been reached by Bang et al [Bang J, Kim S, Kim S, Song J, Kang J, Kim J, et al. Impact of the serum albumin level on acute kidney injury after cerebral artery aneurysm clipping. PLoS One. 2018;13(11):e0206731. [FREE Full text] [CrossRef] [Medline]27]. The relationship between hemoglobin levels and AKI has been widely studied, with anemia being an independent risk factor for AKI [Tseng P, Chen Y, Wang C, Chiu K, Peng Y, Hsu S, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care. 2020;24(1):478. [FREE Full text] [CrossRef] [Medline]28-Kim-Campbell N, Gretchen C, Callaway C, Felmet K, Kochanek P, Maul T, et al. Cell-free plasma hemoglobin and male gender are risk factors for acute kidney injury in low risk children undergoing cardiopulmonary bypass. Crit Care Med. 2017;45(11):e1123-e1130. [FREE Full text] [CrossRef] [Medline]30]. The findings of our study provide compelling evidence that platelet levels act as a protective factor for AKI in patients with CAP. These results align with the previous investigations conducted by Zhang et al [Zhang C, Wang G, Zhou H, Lei G, Yang L, Fang Z, et al. Preoperative platelet count, preoperative hemoglobin concentration and deep hypothermic circulatory arrest duration are risk factors for acute kidney injury after pulmonary endarterectomy: a retrospective cohort study. J Cardiothorac Surg. 2019;14(1):220. [FREE Full text] [CrossRef] [Medline]31], thereby reinforcing the notion that a decreased platelet count is significantly associated with an elevated risk of AKI occurrence.

The prediction process of ML algorithms can be likened to a “black box” [Seetharam K, Balla S, Bianco C, Cheung J, Pachulski R, Asti D, et al. Applications of machine learning in cardiology. Cardiol Ther. 2022;11(3):355-368. [FREE Full text] [CrossRef] [Medline]32], where the inner workings may not be readily understandable to clinical physicians. Therefore, it is crucial for physicians to not solely rely on predictive models but also use their expertise in making appropriate diagnostic and therapeutic decisions. In this study, we used SHAP and LIME techniques to visualize the overall and individual contributions of each indicator to the model’s predictive performance, providing additional insights to clinical physicians. By quantifying the individual risks of AKI occurrence through a web-based prediction platform, we enhanced the clinical utility of the predictive model and demonstrated its significant potential in AKI risk prediction.

In this study, a comprehensive model was developed to predict the occurrence of AKI in patients with CAP by integrating multiple feature variables, resulting in robust predictive performance in both internal and external datasets. In addition, to enhance the interpretability of the best-performing DF model, visual explanations were provided using SHAP and LIME techniques.

Furthermore, a web-based prediction platform was established to augment the clinical utility of the predictive model, offering practical applications in real-world health care settings. The threshold of 74.5% was chosen based on the method of selecting the maximum F₁-score on the training set. In a real-world clinical setting, a model with low sensitivity means that a significant number of patients with AKI might not be identified. This could lead to missed opportunities for early intervention and treatment, potentially worsening patient outcomes. However, the higher PPV suggests that when the model does predict AKI, it is more likely to be correct. This reduces the burden of false positives on clinicians, who might otherwise need to follow up on many non-AKI cases, saving time and resources.

The choice of threshold reflects a preference for higher precision, which might be suitable in situations where the cost of false positives is high, such as in scenarios with limited resources or when unnecessary treatments pose risks to patients. However, the trade-off with lower sensitivity means clinicians must be aware that not all cases of AKI will be caught by the model, and they should continue to use their clinical judgment and other diagnostic tools to ensure comprehensive patient care. Remarkably, the DF model outperformed others in both the internal and external validation cohorts.

However, it is important to acknowledge the limitations of this study. First, it should be noted that this was a retrospective study, which may have introduced certain limitations in terms of the completeness, accuracy, and homogeneity of data recording compared with prospective studies. In addition, the presence of selection bias cannot be ruled out, highlighting the need for further validation through prospective research. Second, the collection of urine output data posed challenges, resulting in cases of incomplete data collection. Furthermore, the consideration of urine output standards was not incorporated in this study, which could have provided additional valuable insights. Finally, while the constructed risk prediction model shows promise, it remains uncertain whether its implementation in clinical practice would translate into tangible benefits for patients. Therefore, prospective and multicenter studies are necessary to comprehensively evaluate the effectiveness and real-world impact of the model. Acknowledging and addressing these limitations will be essential in further refining the understanding and application of the findings in future research.

In summary, this study presents a valuable early prediction tool utilizing ML techniques to identify patients with high risk of AKI among patients with CAP. The developed predictive model has the potential to aid physicians in the early identification of patients at a higher risk of AKI during their hospitalization, enabling prioritized treatment interventions and ultimately improving patient outcomes. Moving forward, our future research endeavors will focus on prospective evaluation of the effectiveness of our AKI prediction model and determining its impact on the prognosis of patients with AKI in real-world clinical practice. In addition, the model could be applied to other etiologies of AKI, such as sepsis-induced AKI, to assess its generalizability.

Conclusion

This study developed a ML-based prediction model for CAP-AKI, demonstrating excellent predictive performance in both internal and external validation cohorts. By integrating easily accessible clinical variables, the DF model achieved high accuracy and interpretability, offering robust support for early risk assessment and targeted intervention in patients with CAP-AKI. In the future, we hope to reduce the risk of CAP-AKI through our online prediction platform.