The Price of Explainability in Machine Learning Models for 100-Day Readmission Prediction in Heart Failure: Retrospective, Comparative, Machine Learning Study

Introduction

Unscheduled rehospitalization has garnered significant research attention due to its high cost and its association with unfavorable patient outcomes [1,2]. In particular, the rehospitalization of patients with heart failure (HF) has been a focal point of the investigation, given the alarmingly high unscheduled readmission rates, which recent studies have reported to be around 30% within 3 months of hospital discharge [3-6]. In the context of value-based care, health care providers are actively working to minimize readmission rates and enhance patient care outcomes. Reducing readmissions involves implementing measures both during a patient’s hospital stay and in the postdischarge phase to ensure adherence to care plans and optimize treatment [3].

Numerous risk factors have been identified as being related to high-risk readmissions in patients with HF. Specifically, there are correlations with serum potassium levels, N-terminal prohormone of brain natriuretic peptide (NT-proBNP), and suboptimal medication adherence [2,3,7]. Moreover, HF is accompanied by several significant comorbidities, including hypertension, diabetes, chronic kidney disease, and atrial fibrillation, all of which impact both the management and prognosis of the condition [2,3].

Machine learning (ML) techniques, especially deep learning (DL), have proven to be highly effective for readmission prediction tasks. This is because the data inputs involved are often intricate and may not be readily discernible by physicians [8-11]. DL methods, in particular, require substantial amounts of data and computational power to uncover latent patterns, eliminating the necessity for domain-specific knowledge. Nevertheless, DL models are often associated with limited explainability owing to their sheer complexity [12]. Conversely, the abundance of variables within electronic health care records (EHRs) poses a challenge for traditional ML methods, often referred to as shallow ML techniques. Consequently, the feature engineering step becomes essential for extracting meaningful features from raw data, often with the assistance of domain-specific knowledge.

Recently, shallow ML has exhibited the potential to surpass state-of-the-art DL techniques when accompanied by effective feature engineering [7-10]. Consequently, the shallow ML approach not only requires less training time but also yields discernible features. Model features would be more straightforward to grasp, offering greater opportunities to interpret the outcomes. Nevertheless, it is crucial that the feature engineering step is performed in collaboration with continuous feedback from clinical experts, ensuring the selection of pertinent variables for modeling the prediction task. Clinicians provide essential guidelines on which data hold clinical significance and how to classify those data. This clinical guidance is crucial for making decisions, such as determining which laboratory values to prioritize and whether transforming continuous values into categories, such as high, normal, or low, can enhance the performance of the ML model.

ML has the potential to aid clinicians in the management of patients with HF at the point of discharge by predicting those at an elevated risk of readmission [13]. However, a lingering question is how to quantify the practical utility of these models in actual clinical practice. Previous studies have demonstrated the potential applicability of their developed models in achieving cost savings [14]. Nonetheless, when it comes to integrating a model into the clinical workflow, it is not just the model’s discrimination ability that holds significance, but also its applicability in real clinical scenarios. To instill confidence in clinicians, it is essential to provide interpretability of the model results, elucidating the individual variables that influence the model’s decisions. This enables clinicians to verify the clinical implications of patient conditions [15].

This study aims to explore the potential of using ML to forecast the risk of readmission due to HF deterioration within 100 days after discharge. This investigation involves a comparison between shallow and deep models applied to real-world health care data. Furthermore, our objective is to enhance the interpretability of these models to ensure they are clinically reliable and actionable.

Methods

Data Source

This study is based on retrospective data from the Regional Health Care Information Platform in Region Halland, an integrated care system located in southwestern Sweden [16]. The system includes data from both the primary and secondary health care levels, including all prescribed medications, clinical examination results (eg, laboratory assessments and radiological examinations), and care delivery resources. Diagnoses (including procedures) and medications were presented according to standard schemas: International Classification of Diseases, revision 10, Sweden (ICD-10-SE) and Anatomical Therapeutic Chemical (ATC) classification system codes, respectively.

Ethics Considerations

The study was approved by the Swedish Ethical Review Board, Stockholm Department 2 Medicine (registration number 2020-00455). Informed consent was waived because this was a retrospective study that the Swedish Ethical Review Board approved. All the methods in this study were performed in accordance with relevant guidelines and regulations.

Study Population

The cohort for this study comprised individuals who had been diagnosed with HF based on ICD-10-SE codes (I11.0, I42, I43, and I50; see Table S4 in Multimedia Appendix 1), were residents receiving care within the Region Halland, and met specific inclusion criteria (ie, aged ≥40 years and had experienced at least one hospital admission after their HF diagnosis between January 1, 2017, and December 31, 2019). We considered all-cause hospitalization. In our study, for each admission within the cohort, we took into account all of the patient’s prior admissions within a 5-year period preceding the current admission (referred to as the lookback period). It is important to note that these previous admissions were not treated as events within our study; instead, they were exclusively used as historical data.

We excluded the following from our analysis: hospitalizations that occurred before the patient’s initial HF diagnosis, hospitalizations of patients younger than 40 years of age at the index admission, hospitalizations in which patients passed away before discharge, and hospitalizations in which patients passed away within 100 days following discharge. Additionally, hospitalizations with a length of stay exceeding 31 days were excluded, as they exhibited a higher likelihood of being influenced by multiple coexisting diseases, as well as factors such as frailty and other medical conditions. The study cohort consisted of 6040 patients, with 2728 (45.02%) being women. These patients collectively contributed to a total of 20,598 hospital admissions. However, after implementing our exclusion criteria, as outlined in Figure S1 in Multimedia Appendix 1, we considered 15,612 (75.79%) of these admissions for our analysis.

Patient Variables and Data Definitions

We collected variables from the Regional Health Care Information Platform that can have a significant influence on readmission prediction as shown in Tables 1-4. The variables considered were patient demographics, comorbidities, medications, and laboratory results. The choice of variables was made through a collaborative process involving both clinical expertise and data science knowledge. For example, we selected comorbidities including hypertension, chronic kidney disease, and atrial fibrillation. We also applied the Charlson Comorbidity Index (CCI) to compute a score associated with different medical comorbid conditions, such as liver disease, diabetes, and obstructive pulmonary disease [17,18]. These comorbidity variables consider the complete patient history, including all visits documented in the EHR system, encompassing specialist and primary care visits. We used ICD-10-SE codes to calculate these comorbidity features, and additional information can be found in Table S2 in Multimedia Appendix 1.

We collected the most recent results of specific laboratory tests conducted before discharge, including NT-proBNP [19], sodium, potassium, ferritin, and estimated glomerular filtration rate. Notably, NT-proBNP values were categorized into normal, gray zone, and elevated ranges [17]. The rest of the laboratory values were categorized into normal and abnormal ranges according to international clinical recommendations (eg, extreme, low, and high), focusing on extracting the correlations between abnormal laboratory values and readmission risk. For the missing laboratory values, we adopted the assumption that they were within the normal range, a methodology consistent with previous studies [20,21]. Furthermore, we gathered information about medications prescribed at the time of discharge, including β-blockers, renin-angiotensin-aldosterone system inhibitors, mineralocorticoid receptor antagonists, loop diuretics, and sodium-glucose cotransporter-2 (SGLT-2) inhibitors. Detailed specifications and ATC codes can be found in Table S3 in Multimedia Appendix 1. In addition, we considered the patient’s medication history by introducing variables to indicate whether each medication had been prescribed at discharge within the previous year for each admission.

We also incorporated a few administrative features related to hospital utilization during the lookback period. These included the total length of stay for inpatient care, the count of outpatient visits (including hospital-associated and primary health care visits), and the number of all-cause hospitalizations. Tables 1-4 display the baseline characteristics of the HF cohort and the engineered features used for training the ML models. It is worth noting that the same patient cohort has been previously investigated, and the clinical variables used in this study have been identified as being linked to the estimation of 100-day readmission risk [22].

The target label, termed “readmission risk,” is defined as a binary variable. A default value of 0 signifies low risk. However, this label is set to 1 if a patient experiences an unplanned admission within 100 days following their discharge. It is important to note that cases in which patients were discharged to another health care facility or in-hospital transfers were not regarded as unplanned readmissions.

Modeling Strategies

We conducted a comparison between the 2 modeling approaches. Our initial model was constructed using CatBoost (Yandex), representing a shallow model developed through traditional ML techniques. CatBoost uses gradient boosting decision trees, renowned for their capacity to handle both categorical and continuous features [23]. The CatBoost model generates predictions by using a series of decision trees, rendering it a more explainable model. By contrast, deep models, specifically recurrent neural networks such as long short-term memory (LSTM) networks [24], have garnered significant attention due to their ability to model the sequential nature inherent in EHRs [7,10,21]. Hence, our second model is constructed as an LSTM network, as depicted in Figure S2 in Multimedia Appendix 1. The LSTM model is designed to learn intricate nonlinear associations between the model variables and outcomes, necessitating the establishment of a surrogate model for providing explanations. In this research, we opted for the LSTM network architecture for DL. Nonetheless, it is worth noting that sequential information regarding patient visits can also be effectively modeled using other recurrent neural network architectures [7].

Figure 1 illustrates the various levels of analysis conducted in this study. As we used 2 distinct data modeling strategies, we organized the data into 2 different formats. For the shallow model, the input data were structured as a single record containing a list of engineered features, computed based on predefined variables outlined in Tables 1-4. Conversely, the input for the DL model comprised raw structured EHR-based data for a sequence of 5 admissions, consisting of the index admission and the 4 prior ones. For each of these admissions, we treated the associated diagnoses as a list of ICD-10-SE codes. Similarly, we regarded procedures, medications, and abnormal laboratory test results as lists in the DL model. However, the shallow model involved an additional preliminary step, the feature engineering phase, in which we computed the values of engineered features. The primary objective of this phase is to convert raw data into a collection of computed features, which are utilized as input for the CatBoost model. Subsequently, we progress through the next 3 phases for both models: input provision, model calibration, and the extraction of explanations for model decisions.

‎

Apart from the models we developed, we also assessed the LACE (Length of stay, Acuity of index admission, CCI, and number of emergency department [ED] visits in the last 6 months) index as an evaluation tool. The LACE index is a tool used for predicting unscheduled readmission, utilizing readily available clinical predictors that can be calculated before a patient’s discharge [25]. The LACE index is formulated using 4 variables to estimate the risk of either death or non-elective readmission following a patient’s discharge from the hospital. These variables are the length of stay, acuity of the index admission, the CCI, and the count of ED visits within the last 6 months [26].

Model Explainability

We used the Shapley Additive Explanations (SHAP) technique to offer insights into model decisions by revealing the most important features for readmission prediction. The SHAP functions as a model-agnostic explanation tool, providing both local and global explanations. The SHAP uses coalitional game theory and offers a model-agnostic approach for computing both the overall behavior of the model and local explanations for specific readmissions [27]. The global explanations furnish a roster of critical features that either contribute to or detract from the risk of readmission. The technique also assigns a score to each prediction through the calculation of additive feature importance. This score can highlight the risk factors that are either positively or negatively influencing the readmission risk for a specific admission. Additionally, clinicians evaluated the explanations offered by each model from a clinical perspective. In particular, clinicians provided feedback on the most critical features identified by SHAP in the context of their clinical assessment of patient conditions. They assessed whether these features could serve as indicators or warnings of undesirable outcomes.

Statistical Analysis

We divided the data into 2 segments. The first subset, spanning from January 1, 2017, to August 31, 2019, consisting of 13,800/15,612 admissions (88.39%), was used for model training. The second, more recent subset, spanning from September 1, 2019, to December 31, 2019, containing 1812/15,612 admissions (11.61%), was reserved for holdout testing. We fine-tuned the hyperparameters of each model using the grid search method, which conducts a thorough exploration of a predefined set of hyperparameter combinations. To train the models, we implemented a stratified 10-fold cross-validation approach, where the training data were subdivided into 10 segments. In each iteration, a fold consisting of randomly chosen 90% (12,420/13,800) admissions of the training data was used for model training, while the remaining admissions (1380/13,800, 10%) were used for validation. This process was repeated 10 times, with a different 10% of the data reserved for validation in each iteration. We used a sensitivity-based training strategy to optimize the sensitivity of the trained models, making them particularly relevant in clinical operations. Of the 10-fold cross-validation training process, we selected the model that achieved the highest sensitivity while maintaining a minimum specificity of 50%.

The assessment of predictive model performance is founded on widely used performance metrics, including sensitivity, specificity, F₁-score, receiver operating characteristic curves, the area under the receiver operating characteristic curve (AUC), and the area under the precision-recall curve, all of which are reported with 95% CIs. Additionally, we calculated the number needed to screen, the number needed to diagnose, and the number needed to predict [28].

Results

Patient Variables and Data Definitions

The features of admissions, encompassing both unscheduled readmissions and planned admissions within the following 100 days after discharge, are detailed in Tables 1-4. As indicated, the instances of readmitted patients exhibited a notably higher prevalence of comorbidities in comparison to those cases that did not result in readmission.

Model Evaluation

Table 5 presents the performance achieved by the models, with the characteristics of the training and holdout test cohorts detailed in Table S1 in Multimedia Appendix 1. It is worth noting that CatBoost outperformed LSTM in both the validation and holdout test sets, achieving superior sensitivity and specificity. The sensitivity-based training approach led to an increased number of false positives, which are misclassified admissions labeled as unplanned readmissions, in both models, consequently resulting in lower specificity. Concerning the other metrics, the performance of both models is nearly identical. In both cases, the numbers needed to screen/diagnose/predict were approximately 4-5. Notably, the models we developed exhibited superior performance when compared with the LACE index. We used a threshold of 10 to identify high-risk readmissions based on the LACE scores index [26].

As aforementioned, we opted for sensitivity over specificity to be relevant in clinical operations, thus, our models generate more false positives than the LACE index. The LACE index is not trainable compared with the proposed CatBoost and LSTM models, yet it is commonly used to estimate readmission risks in general. However, a study on patients with HF showed that high LACE scores were associated with higher rates of ED revisits after hospital discharge but they had less accuracy in predicting readmissions [25].

The lower specificity ratios observed for both LSTM and CatBoost models are attributed to the sensitivity-based training approach we used. This approach prioritizes achieving a higher sensitivity, and as a result, accepting a lower specificity (with a minimum of 0.50) is part of the strategy to achieve this goal.

Model Explainability

Figure 2 provides an overview of the variables of importance identified by the CatBoost and LSTM models, categorized into several main groups. Given that we used 2 distinct input formats for these models, our interest lies in understanding the commonalities and disparities in what they have identified. Both models concurred in identifying the most critical variables for predicting readmission. These variables included a higher number of previous readmissions, emergency admissions, longer lengths of stay, and age. By contrast, the history of medications was assigned a lower level of importance by both models. Furthermore, both models highlighted extreme values of NT-proBNP as an influential variable pushing toward readmission. Additionally, both models indicated that patients prescribed loop diuretics at discharge were at a higher risk of readmission. However, CatBoost identified angiotensin-converting enzyme inhibitors, mineralocorticoid receptor antagonists, β-blockers, and angiotensin receptor neprilysin inhibitor treatments at discharge as factors associated with a lower risk of readmission.

‎

For the CatBoost model, the CCI was identified as the most critical feature. This feature, however, was not available for the LSTM model. Conversely, the LSTM model placed greater importance on diagnosis codes compared with other clinical information such as abnormal laboratory test results and medication codes. Interestingly, in both models, patient age was associated with an increased risk of readmission, while the duration of HF appeared to reduce this risk. You can find the top 28 features highlighted by both models in Figures S3 and S4 in Multimedia Appendix 1.

Local explanations are valuable for enabling health care professionals to comprehend the rationale behind model predictions and make informed decisions. In Figure 3, local explanations are provided for a particular admission where both models correctly classified it as a high-risk readmission. In the case presented, the 2 most important features were identical in both models: the number of previous unscheduled readmissions and the number of ED visits. Additionally, both models found that the presence of chronic kidney disease and abnormal laboratory test values of NT-proBNP (indicative of HF) contributed to an increased risk of readmission. Conversely, a short length of stay, specifically 1 day in this instance, exerted a negative impact on the risk of readmission. Additional examples, where the models disagree on the target label, can be found in Figures S5 and S6 in Multimedia Appendix 1.

‎

Discussion

Principal Findings

Our primary motivation for this study is to develop models that can effectively adapt to patterns within patients with HF data. Given the absence of prior research demonstrating that ML models can substantially reduce readmission, our study focuses on the comparison of shallow and DL models in the context of patients with HF data. This study encompassed a large HF cohort, with a readmission rate of 5597/15,612 (35.85%) per year, which aligns with findings from comparative studies [3,6,22]. Notably, the more interpretable shallow model exhibited superior performance compared with the deep model, and both models significantly outperformed conventional scores in terms of the numbers needed to screen for identifying readmission within 100 days. Consequently, this study does not suggest that achieving enhanced prediction performance necessitates sacrificing model transparency.

The results indicate that the shallow model, while more interpretable, performs as effectively as the deep model. However, it is important to note that the shallow model necessitates a feature engineering process to derive useful features from raw EHR data. By contrast, the input to the deep model closely resembles the original EHR data, enabling a comparison of the patterns learned from raw data against those generated by engineered features calculated with the guidance of expert knowledge. Interestingly, the explainability results reveal that there is an overlap in the most critical features identified by both models.

Good explanations can serve as an advantage in building trust among clinicians and guiding interventions. Understanding the features that influenced the prediction is crucial, as previous reports have emphasized the importance of comprehending the models to establish trust in them [14,15]. The selection of variables used for model training was made to support generalizability and the potential transfer of the model to new settings. ICD-10-SE codes, ATC codes, and standard laboratory values are expected to be available in most acute care venues, enhancing the applicability of the model.

Model Explainability and Clinical Insights

In addition to clinically informative variables, administrative features exhibit significant importance in both models, consistent with findings from prior studies [8,11,29]. It appears that a history of previous hospital admissions, readmissions, and visits to the ED are associated with an elevated risk of readmission. Both models underscore the connection between patients with frequent inpatient episodes, often indicative of a high disease burden, and features that are not easily altered, such as age and comorbidity. Nevertheless, these features serve as crucial warning signals that should be considered when planning hospital discharge. As a result, the feature engineering process in this study was continually refined to encompass additional clinically relevant features. Given that previous readmissions and ED visits are robust indicators of potential new events, the factors contributing to past unplanned health care encounters could serve as key targets for action. Consequently, further investigation into these factors, with an emphasis on causal inference, is warranted to improve patient outcomes and reduce readmission rates.

Previous studies have demonstrated the positive impact of HF medication in preventing readmission [6]. In addition to medical treatment, factors such as continuity and availability of care are valuable tools for these patients. Insights revealing weak associations, such as the finding that guideline-compliant pharmaceutical treatment is only weakly linked to readmission risk, should emphasize the importance of focusing on nonmedical strategies to prevent readmissions alongside optimizing drug regimens, with an emphasis on drugs known to reduce readmission rates [30-32]. We should also not overlook previous findings that sodium and potassium levels, as well as NT-proBNP, are associated with the outcomes, as confirmed in this report. Moreover, the use of β-blockers and other first-line treatments can have an impact on patient outcomes. These factors should continue to be considered in patient care and intervention strategies [2,3,6,7].

Our focus on readmission serves a dual purpose: it acts as an indicator of health outcomes and, simultaneously, as an encounter that contributes to resource utilization, potentially conflicting with other health care priorities. The need to screen (and subsequently intervene with) approximately 4.5 patients to identify and target 1 potentially preventable readmission should be recognized as a compelling reason for further interventional studies, where the cost of the intervention is assessed. One potential intervention approach could involve care planning tailored to the identified drivers of readmission probability as determined in this study.

Limitations

In this study, we did not encompass aspects of care beyond hospitals and primary health care. As readmissions were only weakly associated with pharmaceutical treatment and several other predictive factors were not easily amenable to intervention, strategies to reduce readmissions may need to concentrate on home care and enhancing patient comfort. However, it is important to note that these aspects were not within the scope of this study.

Moreover, the number of patients on SGLT-2 inhibitors in our study was too small to draw definitive conclusions about their impact on readmissions. However, other studies suggest that this medication warrants further investigation in real-world health care settings. It is important to emphasize that the effectiveness of treatment with drugs such as SGLT-2 inhibitors is not questioned according to guidelines and remains one of the major tools for improving patient outcomes.

Comparison With Prior Work

A broad spectrum of predictive modeling methods, including statistical approaches and conventional ML and DL methods, have been used to tackle the readmission prediction problem [14]. There is a prevailing trend in the literature toward the utilization of traditional statistical and conventional ML models. These models are favored because they offer greater transparency, are easier to understand, and have been shown to outperform DL models in certain contexts [8,9,25]. Prior studies have often concentrated on comparing the predictive performance of different models, often neglecting to investigate their interpretability and practical usefulness in real clinical settings. However, with the rapid advancements in explainable artificial intelligence and the growing adoption of model-agnostic interpretation techniques, even complex DL models can offer both local and global explanations, as demonstrated in this study.

Conclusions

In clinical prediction models, there can be a trade-off between precision and explainability. When models exhibit similar prediction performance, the less opaque or more interpretable model is often favored. In the presented case, the shallow ML model performs at a similar level to a deeper model and significantly outperforms traditional scoring methods. Achieving explainability requires the collaborative effort of clinicians through feature engineering. However, it is worth noting that many of the features driving the predictions in this study may not be easily actionable. Enhancing compliance with treatment guidelines can lead to improvements, but for more substantial impacts, exploring new drugs and interventions beyond the scope of this study should also be considered.