Developing a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study

Introduction

Major adverse cardiac and cerebrovascular events (MACCE) are among the leading causes of perioperative morbidity and mortality following noncardiac surgeries, particularly in an aging population [Smilowitz NR, Gupta N, Ramakrishna H, Guo Y, Berger JS, Bangalore S. Perioperative major adverse cardiovascular and cerebrovascular events associated with noncardiac surgery. JAMA Cardiol. 2017;2(2):181-187. [FREE Full text] [CrossRef] [Medline]1-Writing Committee for the VISION Study Investigators, Devereaux PJ, Biccard BM, Sigamani A, Xavier D, Chan MTV, et al. Association of postoperative high-sensitivity troponin levels with myocardial injury and 30-day mortality among patients undergoing noncardiac surgery. JAMA. 2017;317(16):1642-1651. [CrossRef] [Medline]4]. With over 300 million noncardiac surgeries performed annually, accurate preoperative risk assessment has become essential to optimize patient outcomes and reduce health care costs [van Klei WA, Grobbee DE, Rutten CLG, Hennis PJ, Knape JTA, Kalkman CJ, et al. Role of history and physical examination in preoperative evaluation. Eur J Anaesthesiol. 2003;20(8):612-618. [CrossRef] [Medline]5,Alkire BC, Raykar NP, Shrime MG, Weiser TG, Bickler SW, Rose JA, et al. Global access to surgical care: a modelling study. Lancet Glob Health. 2015;3(6):e316-e323. [FREE Full text] [CrossRef] [Medline]6]. However, the predictive accuracy of traditional assessment tools is not consistently high, and various tools are used at different physicians’ discretion [Cohen ME, Bilimoria KY, Ko CY, Richards K, Hall BL. Effect of subjective preoperative variables on risk-adjusted assessment of hospital morbidity and mortality. Ann Surg. 2009;249(4):682-689. [CrossRef] [Medline]7].

Traditionally, the Revised Cardiac Risk Index (RCRI), which comprises 6 equally weighted components, is extensively used to mitigate major perioperative cardiac complications owing to its simplicity and relatively high predictability of in-hospital major adverse cardiac events (MACE) or cardiovascular-related death [Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100(10):1043-1049. [FREE Full text] [CrossRef] [Medline]8]. However, the index developed over 2 decades ago has certain challenges, including limited external validation and reduced precision in vascular surgery [Gupta PK, Gupta H, Sundaram A, Kaushik M, Fang X, Miller WJ, et al. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124(4):381-387. [CrossRef] [Medline]9]. These factors may modestly impact its effectiveness in predicting clinical outcomes following noncardiac surgeries in practical clinical environments [Brasher PMA, Beattie WS. Adjusting clinical prediction rules: an academic exercise or the potential for real world clinical applications in perioperative medicine? Can J Anaesth. 2009;56(3):190-193. [CrossRef] [Medline]10]. Subsequent predictive tools, such as the American College of Surgeons, National Surgical Quality Improvement Project (NSQIP), and NSQIP Myocardial Infarction or Cardiac Arrest, developed after RCRI, also show strong performance in predicting postoperative MACE. However, these tools pose challenges for clinicians in practical clinical use because they rely on subjective predictors, leading to low interrater reliability [Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833-42.e1. [FREE Full text] [CrossRef] [Medline]11]. Despite their enhanced predictive accuracy, their application in real-world settings is often constrained by these practical limitations. Given these challenges, our research uses machine learning techniques integrated with the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). Recent advances in machine learning have demonstrated significant potential in addressing these limitations by leveraging large-scale electronic health records (EHRs) to develop predictive models with enhanced accuracy and adaptability. Machine learning algorithms can extract meaningful patterns from high-dimensional datasets, facilitating the identification of key predictors for perioperative risks [Sun H, Depraetere K, Meesseman L, Cabanillas Silva P, Szymanowsky R, Fliegenschmidt J, et al. Machine learning-based prediction models for different clinical risks in different hospitals: evaluation of live performance. J Med Internet Res. 2022;24(6):e34295. [FREE Full text] [CrossRef] [Medline]12]. Furthermore, the OMOP CDM standardizes diverse observational databases, improving data interoperability and facilitating seamless integration of predictive models across institutions. This standardized framework enhances data sharing and model validation across health care systems, ensuring broader applicability and reliability, as highlighted by Ahmadi et al [Ahmadi N, Nguyen QV, Sedlmayr M, Wolfien M. A comparative patient-level prediction study in OMOP CDM: applicative potential and insights from synthetic data. Sci Rep. 2024;14(1):2287. [FREE Full text] [CrossRef] [Medline]13] in their evaluation of OMOP CDM’s transformative potential in harmonizing patient data across institutions [Ahmadi N, Nguyen QV, Sedlmayr M, Wolfien M. A comparative patient-level prediction study in OMOP CDM: applicative potential and insights from synthetic data. Sci Rep. 2024;14(1):2287. [FREE Full text] [CrossRef] [Medline]13-Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the observational medical outcomes partnership. Ann Intern Med. 2010;153(9):600-606. [CrossRef] [Medline]16].

Compared with traditional tools like RCRI, our model incorporates a significantly larger number of predictors, allowing for a more precise risk assessment. The OMOP CDM framework further enhances this capability by offering a comprehensive and standardized approach to data integration, addressing the limitations of previous models and ensuring adaptability across diverse clinical environments. Building upon this robust foundation, we developed a machine learning–based prediction model that leverages advanced algorithms to analyze complex patterns within extensive patient datasets. Unlike American College of Surgeons, NSQIP, and NSQIP Myocardial Infarction or Cardiac Arrest, which often rely on subjective inputs and are constrained by interrater variability, our model automates predictor integration, ensuring consistency and practicality in real-world applications [Meskó B, Görög M. A short guide for medical professionals in the era of artificial intelligence. NPJ Digit Med. 2020;3:126. [FREE Full text] [CrossRef] [Medline]17,Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347-1358. [CrossRef] [Medline]18]. Through this approach, we aim to provide a more advanced and precise tool for personalized risk prediction, demonstrating improved performance compared to traditional and contemporary predictive models.

Methods

Data Sources

The data sources used in this study were selected and standardized to ensure the integrity and compatibility of the collected information. The EHRs were converted to the OMOP CDM, and source codes were mapped to standard vocabularies, including the Systematized Nomenclature Of Medicine Clinical Terms (SNOMEDCT) [Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54-60. [FREE Full text] [CrossRef] [Medline]19,Reps JM, Schuemie MJ, Suchard MA, Ryan PB, Rijnbeek PR. Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data. J Am Med Inform Assoc. 2018;25(8):969-975. [FREE Full text] [CrossRef] [Medline]20]. Data analysis was conducted using the observational health data sciences and informatics (OHDSI) open-source patient-level prediction (PLP) package, which was purposefully designed for standardized analysis and harmonization with the OMOP CDM. These specialized tools have facilitated efficient data processing and analysis across different datasets within the OHDSI data network, and our study strictly follows the guidelines about machine learning predictive models in biomedical research [FitzHenry F, Resnic FS, Robbins SL, Denton J, Nookala L, Meeker D, et al. Creating a common data model for comparative effectiveness with the observational medical outcomes partnership. Appl Clin Inform. 2015;6(3):536-547. [FREE Full text] [CrossRef] [Medline]21]. This collaborative aspect enhances the comparability and generalizability of the prediction models, making them applicable to diverse health care settings. To address potential data loss or variation during the conversion and mapping process, the PLP package uses a systematic approach to generate training data. Covariates that could not be mapped (concept_ID = 0) are excluded from the input data. Subsequently, a sparse matrix is initialized to represent patient-level covariates and infrequently observed covariates—those with a nonzero frequency below a predefined threshold (default 0.1%)—are excluded to reduce noise. Normalization is then performed by scaling covariates to their maximum observed values, and feature selection techniques are applied to retain only meaningful variables for model training. These steps minimize the impact of unconverted or missing data, ensuring the robustness and reliability of the models.

To develop and evaluate our prediction models, we retrospectively used patient data from 2 tertiary hospitals, Seoul National University Bundang Hospital (SNUBH) and Asan Medical Center (AMC), which are recognized for their substantial CDM datasets. The SNUBH dataset contains data from 46,225 patients who underwent noncardiac surgery between January 2003 and December 2020, and the AMC dataset includes data from 396,424 patients who underwent noncardiac surgery between January 2010 and December 2020. This extensive dataset included a comprehensive array of demographic information and detailed preoperative baseline characteristics, including diagnosis codes, underlying diseases, laboratory test results, medications, type of surgery, and clinical outcomes from the EHR system (Table 1).

Ethical Considerations

This retrospective, observational network study was conducted by a multidisciplinary team comprising cardiologists, medical informatics specialists, and data scientists. The study received approval from the institutional review boards (IRBs) of SNUBH (IRB number 2208-772-906) and AMC (IRB number 2022-1547). Due to the retrospective study design and the use of deidentified data, the requirement for written informed consent was waived.

Study Design and Target Cohort

We conducted a retrospective analysis of patients aged 65 years and older who underwent noncardiac surgeries at 2 independent tertiary hospitals. Age was determined at the time of surgery. We excluded patients who had undergone cardiac or emergency surgery within 3 days of a hospital visit and those who did not have a sufficient observation period of less than 30 days (Figure 1). The prediction time (t=0) and start date of the time-at-risk window for prediction were set as surgery dates. The end date of the time-at-risk window for clinical outcomes was 30 days after surgery. The data collection period for the predictors was defined as 3-365 days before the start date of the time-at-risk window (Figure 2). We adjusted the observational time frame to collect baseline characteristics and preoperative laboratory measurements within a narrower window of 3-30 days before the onset of the time-at-risk period. This adjustment ensured that the data accurately represented the patient's condition at the beginning of the time-at-risk period.

‎

‎

Clinical Outcomes

The clinical outcome of this study was MACCE within 30 days of noncardiac surgery. The individual components of MACCE include myocardial infarction, cardiac arrest or shock, heart failure, stroke, and death. All clinical events were identified and extracted from CDM data using standardized concept IDs (Table S1 in

Multimedia Appendix 1

Supplementary tables and figures.

Prediction Model Development and Validation

Using observational health care data, we used the standardized, open-source OHDSI PLP package within R (version 4.1.0; R Foundation for Statistical Computing) to develop and validate our prediction model. We developed a prediction model by integrating data from preoperative laboratory measurements 16 routinely measured basic parameters: white blood cell, hemoglobin, platelet count, aspartate aminotransferase, alanine aminotransferase, blood urea nitrogen, creatinine, albumin, calcium, sodium, phosphate, total bilirubin, c-reactive protein, cholesterol, hemoglobin A_1c, and prothrombin time), previous diagnosis, medication records, and surgical type from the SNUBH CDM development dataset. This dataset was divided into a training set (34,670/46,225, 75%) and a testing set (11,555/46,225, 25%) for internal validation of the developed model. For the training dataset, we used a 3-fold cross-validation for hyperparameter optimization. Cross-validation was used to minimize overfitting and optimize the model’s generalization capabilities by evaluating its performance on different data splits.

With the OHDSI PLP framework, the least absolute shrinkage and selection operator, logistic regression, gradient boosting machines, AdaBoost, random forest (RF), and decision trees were developed. Model discrimination was assessed using the areas under the receiver operating characteristic curve (AUROC) and areas under the precision-recall curve. In addition, a calibration plot analysis was used to gauge the reliability of the model’s predictions and to confirm that the predicted probabilities matched the actual occurrence probabilities. Finally, the model’s generalizability is evaluated by performing external validation using the AMC CDM dataset. The external validation results demonstrated minimal differences in AUROC values compared to internal validation, confirming the model's ability to generalize effectively to unseen data and supporting the robustness of overfitting prevention strategies such as feature selection, regularization, and cross-validation. To evaluate the relative importance of covariates in developing the prediction model, feature importance was calculated and sorted in descending order based on the most effective machine learning algorithm contributing to the prediction. In addition, we attempted to develop prediction models by recombining variables based on covariate grouping. These recombination models were developed by excluding certain covariate groups with the expectation that this selective process may enhance prediction accuracy by reducing noise and focusing on the most significant factors.

Results

Study Population

A total of 46,225 patients were enrolled at SNUBH, and 396,424 were enrolled at AMC, with an average age of 72.9 (SD 5.35) years at both hospitals (Table 1). More male than female patients were enrolled at both institutions, with 25,573/46,225 males (55.3%) at SNUBH and 232,522/396,424 males (58.7%) at AMC. Hypertension was the most common comorbidity, affecting 28,641/46,225 (62%) and 216,440/396,424 (54.6%) of patients at SNUBH and AMC, respectively. At SNUBH, cerebrovascular disease was more common (7363/46,225; 15.9%) compared with 40,628 of 396,424 (10.2%) at AMC. By contrast, congestive heart failure was more common at AMC (19,613/396,424, 4.9%) than at SNUBH (961/46,225, 2.1%). Similarly, ischemic heart disease had a higher representation at AMC (43,302/396,424, 10.9%) than at SNUBH (3939/46,225, 8.5%). Preoperative laboratory results were within normal ranges, with patients with a creatinine level of 2 mg/dL or higher accounting for 3434 of 46,225 (7.4%) at SNUBH and 74,594 of 396,424 (18.8%) at AMC. Regarding medications, the AMC data showed a higher proportion of patients registered with aspirin, P2Y12 inhibitors, beta-blockers, renin-angiotensin system inhibitors, calcium channel blockers, statins, and insulin treatment.

There was a significant difference between the 2 hospitals regarding the type of surgery and post-noncardiac surgery MACCE within 30 days across all categories. Surgeries with a risk exceeding 1% are presented in Table 1; those with unmapped names were classified as unspecified. Post-noncardiac surgery MACCE within 30 days included myocardial infarction, which occurred in 907 of 46,225 (2%) patients at SNUBH and 5603 of 396,424 (1.4%) at AMC, heart failure in 308 of 46,225 (0.7%) and 2310 of 396,424 (0.6%), and strokes in 799 of 46,225 (1.7%) and 6017 of 396,424 (1.5%), respectively. In-hospital deaths accounted for 419 of 46,225 (0.9%) and 11,875 of 396,424 (3%) deaths at SNUBH and AMC, respectively.

Prediction Model Performance

The predictability of prediction models for internal and external validation is presented in Table 2. The numbers of patients included in the training, test, and external validation sets of the SNUBH model who met the inclusion criteria are presented in Table S2 (

Multimedia Appendix 1

Supplementary tables and figures.

Figure 3

Figure 3

Table 2

Figure 3

Figure 3

‎

The superior performance of the RF model can be attributed to its unique horizontal ensemble structure that uses bagging to construct decision trees based on randomly selected subsets of features at each split. This structure minimizes tree correlation, reduces overfitting, and handles high-dimensional low-sample size datasets, which are characteristic of electronic medical record data. Furthermore, RF is robust to imbalanced data, outperforming models like gradient boosting machines in scenarios with severe class imbalance. Gradient boosting machines, in contrast, use a boosting structure that sequentially trains weak learners, making them sensitive to noise and rare events and highly dependent on optimal hyperparameter tuning. Compared with simpler models like logistic regression and LASSO, RF excels in capturing complex patterns in high-dimensional data with many irrelevant or noisy features, making it particularly suitable for electronic medical record datasets [Ahmadi N, Nguyen QV, Sedlmayr M, Wolfien M. A comparative patient-level prediction study in OMOP CDM: applicative potential and insights from synthetic data. Sci Rep. 2024;14(1):2287. [FREE Full text] [CrossRef] [Medline]13,Blagus R, Lusa L. Gradient boosting for high-dimensional prediction of rare events. Computational Statistics & Data Analysis. 2017;113:19-37. [CrossRef]22-Mienye ID, Sun Y. A survey of ensemble learning: concepts, algorithms, applications, and prospects. IEEE Access. 2022;10:99129-99149. [CrossRef]24].

Predictors

In the prediction model, we assessed the relative importance of various covariates based on their values (Figure 4). Rather than identifying a single outstanding covariate, the analysis grouped covariates into similar thematic clusters. Predominantly, predictors associated with the patient’s underlying medical history were relatively high in the developed prediction model. These include ischemic heart disease, traumatic and nontraumatic brain injury, heart failure, heart disease, and cerebral infarction. The model highlights the importance of the measurement predictors. Preoperative laboratory measurements revealed that hemoglobin, creatinine, albumin, CK-MB, and erythrocyte sedimentation rates played crucial roles. Among the medication predictors, antithrombotic agents and beta-blockers were notably prominent, whereas the significance of the others was less pronounced. Furthermore, although important, the significance of the type of surgery did not appear to be as substantial as expected when compared with other factors in the model.

‎

In addition, we developed prediction models by recombining the data and considering previous diagnoses, medication, type of surgery, and measurement data in various combinations (Table S3 in

Multimedia Appendix 1

Supplementary tables and figures.

Discussion

Principal Findings

In this study, we developed and evaluated an advanced perioperative risk prediction model using a CDM-based machine learning approach. The results demonstrated that machine learning models consistently outperformed traditional methods, such as the RCRI score, in predictive accuracy. For instance, the RCRI score achieved an AUROC of 0.704, whereas the RF model, among 5 tested machine learning models, showed the best overall performance with an AUROC of 0.897 for internal validation and 0.817 for external validation. These findings highlight the robustness and generalizability of the model across diverse datasets and outcomes.

This study provides key insights into the potential of CDM-based machine learning to enhance clinical predictive modeling. By achieving superior predictive accuracy and scalability, especially in external validation, our approach demonstrates a promising pathway for developing reliable tools for perioperative risk prediction across institutions. Advances in machine learning for extensive dataset analysis have led to increased interest in applying PLP and offer the potential for medical practice to consider personalized risks as part of clinical decision-making [Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc. 2017;24(1):198-208. [FREE Full text] [CrossRef] [Medline]25]. The adoption of the OMOP CDM has streamlined the transformation of diverse concept domains, encompassing medical conditions, drugs, procedures, and measurements derived from health record systems or reported information into labeled analytic data. This transformation ensures semantic and syntactic interoperability, enhancing the extraction of prediction variables and facilitating seamless integration across various health care data sources [Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54-60. [FREE Full text] [CrossRef] [Medline]19,Hripcsak G, Duke JD, Shah NH, Reich CG, Huser V, Schuemie MJ, et al. Observational health data sciences and informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574-578. [FREE Full text] [Medline]26]. The expanding adoption of the OMOP CDM across health care institutions globally further strengthens the transferability of predictive models. For example, over 60 databases in South Korea, covering approximately 73 million patients, have been converted to the OMOP CDM format. This widespread implementation promotes interoperability, cross-institutional research, and scalability of predictive models in diverse health care environments. In addition, the standardized data across different institutions allowed a fair evaluation of the predictive performance of the models by extensive external validation [Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347-1358. [CrossRef] [Medline]18]. To implement this framework, we used the OHDSI “Patient-Level Prediction” package, which integrates seamlessly with the OMOP CDM and offers significant advantages. This package not only ensures model reproducibility and transparency through its open-source nature but also provides flexibility in choosing machine learning algorithms and feature engineering techniques. Furthermore, its capability for internal and external validation aligns with best practices, promoting robust performance evaluation and generalizability [Reps JM, Schuemie MJ, Suchard MA, Ryan PB, Rijnbeek PR. Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data. J Am Med Inform Assoc. 2018;25(8):969-975. [FREE Full text] [CrossRef] [Medline]20]. Therefore, our model supports existing preoperative evaluation guidelines and enables open dissemination that can be extensively validated across OHDSI collaborator networks.

The well-structured and labeled dataset improves algorithms in supervised machine learning but sometimes leads to overfitting, which prevents the model’s generalization to fit the observed data well [Junqué de Fortuny E, Martens D, Provost F. Predictive modeling with big data: is bigger really better? Big Data. 2013;1(4):215-226. [CrossRef] [Medline]27,Ying X. An overview of overfitting and its solutions. J Phys: Conf Ser. 2019;1168(2):022022. [CrossRef]28]. To overcome the challenge of overfitting, we used a feature selection method as one of several techniques to identify and prioritize factors essential for the learning process [Hawkins DM. The problem of overfitting. J Chem Inf Comput Sci. 2004;44(1):1-12. [CrossRef] [Medline]29,Bagherzadeh-Khiabani F, Ramezankhani A, Azizi F, Hadaegh F, Steyerberg EW, Khalili D. A tutorial on variable selection for clinical prediction models: feature selection methods in data mining could improve the results. J Clin Epidemiol. 2016;71:76-85. [CrossRef] [Medline]30]. In addition, we used a feature selection method to create new combinations of thematic clusters, including medical conditions, drugs, types of procedures, and measurements, to assess their relative importance in predicting adverse outcomes following surgery. The recombination model, which included past medical conditions and previous laboratory data, exhibited a notably high predictive accuracy. Our model’s ability to discern the varying importance of these factors in real clinical contexts underscores the importance of focusing on patient histories and prior laboratory results during preoperative evaluations [Michota FA, Frost SD. The preoperative evaluation: use the history and physical rather than routine testing. Cleve Clin J Med. 2004;71(1):63-70. [FREE Full text] [CrossRef] [Medline]31]. This approach aligns with physicians’ subjective assessments in clinical settings and provides a flexible alternative to traditional methods that may not fully accommodate each patient’s unique circumstances [Glance LG, Faden E, Dutton RP, Lustik SJ, Li Y, Eaton MP, et al. Impact of the choice of risk model for identifying low-risk patients using the 2014 American College of cardiology/American Heart association perioperative guidelines. Anesthesiology. 2018;129(5):889-900. [CrossRef] [Medline]32].

The practical implications of our research extend to potential time and cost savings in clinical settings. Risk assessments often lead to unnecessary procedures or examinations, such as echocardiography, cardiac computed tomography, and cardiac stress tests, being performed on patients [Johansson T, Fritsch G, Flamm M, Hansbauer B, Bachofner N, Mann E, et al. Effectiveness of non-cardiac preoperative testing in non-cardiac elective surgery: a systematic review. Br J Anaesth. 2013;110(6):926-939. [FREE Full text] [CrossRef] [Medline]33]. These tests, even when not closely associated with the patient’s postsurgical outcomes, contribute to ongoing wastage in overall medical costs [Bryson GL, Wyand A, Bragg PR. Preoperative testing is inconsistent with published guidelines and rarely changes management. Can J Anaesth. 2006;53(3):236-241. [CrossRef] [Medline]34-Ferrando A, Ivaldi C, Buttiglieri A, Pagano E, Bonetto C, Arione R, et al. Guidelines for preoperative assessment: impact on clinical practice and costs. Int J Qual Health Care. 2005;17(4):323-329. [CrossRef] [Medline]36]. Several studies have demonstrated that predictive models can effectively reduce unnecessary preoperative testing and associated costs. For example, standardized preoperative models have shown significant reductions in coagulation and renal panel tests, leading to improved resource use without compromising patient safety. Similarly, machine learning–based tools, such as MySurgeryRisk (Azra Bihorac, University of Florida), have enhanced risk stratification for postoperative complications, thereby minimizing the need for unnecessary evaluations. These findings highlight the potential of predictive models to address inefficiencies and optimize preoperative care. Our model, with its high predictive accuracy, is poised to reduce the number of unnecessary tests performed and contribute to medical cost savings. Furthermore, our model could reduce waiting times for patients as unnecessary consultations and tests may be minimized, ultimately mitigating the challenges posed by health care system congestion and assisting patients in undergoing surgery at an appropriate time. In the future, with precise preoperative predictability, we plan to use our model to proactively identify individuals at risk of postsurgical complications and ensure appropriate postoperative management. In an aging population, where surgical mortality and morbidity rates are increasing [Weiser TG, Haynes AB, Molina G, Lipsitz SR, Esquivel MM, Uribe-Leitz T, et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet. 2015;385 Suppl 2:S11. [CrossRef] [Medline]37], this approach can serve as a viable solution to effectively mitigate these challenges.

Strengths

By using the OMOP CDM framework for data standardization, we ensured syntactic and semantic interoperability across diverse datasets. Despite the inherent inconsistencies and missing values often observed in large health care datasets, we mitigated these issues by leveraging data from 2 of the largest tertiary hospitals in South Korea, where the data quality and quantity were sufficient to minimize noise and missing data. This rich dataset enabled the development of a robust machine learning model with high predictive accuracy for perioperative risk assessment.

Furthermore, external validation using datasets from independent institutions demonstrated minimal performance differences compared with internal validation, suggesting that the semantic gap was relatively small. This highlights the model’s strong generalizability and supports its applicability across multiple hospitals. These findings align with previous research emphasizing the importance of model calibration for diverse clinical settings and the need for strategies that address cross-institutional variability without requiring site-specific data harmonization [Sun H, Depraetere K, Meesseman L, Cabanillas Silva P, Szymanowsky R, Fliegenschmidt J, et al. Machine learning-based prediction models for different clinical risks in different hospitals: evaluation of live performance. J Med Internet Res. 2022;24(6):e34295. [FREE Full text] [CrossRef] [Medline]12,Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18. [FREE Full text] [CrossRef] [Medline]38].

In addition, the model consistently outperformed traditional tools, such as the RCRI score, achieving high AUROC values in both internal and external validations. The inclusion of comprehensive preoperative variables, including laboratory results, medications, and comorbidities, provided a personalized approach to risk prediction. This capability has the potential to reduce unnecessary preoperative testing, streamline clinical decision-making, and improve resource allocation in real-world health care settings.

Limitations

This study has several limitations. First, the use of datasets from 2 tertiary hospitals introduced variability in patient populations and clinical practices, which could have influenced model performance. However, this diversity reflects real-world conditions and likely contributed to the model’s robustness, as demonstrated by consistent results in external validation. Second, although the model achieved high AUROC values, the relatively low incidence rate suggests challenges in handling imbalanced outcomes. Feature selection was used to prioritize significant predictors, and future studies may incorporate advanced techniques to address this limitation. Third, the model has not yet been tested in real-time clinical workflows, where factors such as delayed data entry could impact performance. The use of standardized OMOP CDM data ensures scalability and future studies will focus on prospective validation in live clinical settings. Finally, the exclusion of certain covariates, such as frailty scores and socioeconomic status, may have limited the model’s predictive accuracy. To maintain scalability across institutions, the study prioritized universally available predictors, but future work will explore integrating additional variables to enhance performance.

Conclusions

In this study, we successfully developed a high-performance machine learning–based preoperative prediction model by using the standardized data format of the OMOP CDM. This approach offers the potential for improved clinical decision-making and extensive external validation across health care institutions. In the future, our research has practical implications for potential time and cost savings in clinical settings by reducing unnecessary procedures, tests, and consultations, ultimately addressing health care system congestion and improving patient surgical timing.