This paper is in the following e-collection/theme issue:

Artificial Intelligence (1421) Clinical Information and Decision Making (1434) Machine Learning (1501) Coronary Heart Disease (81) Congenital Heart Disease (5) Cardiac Diagnostics (15) New Diagnostic Tools and Instruments for Pediatrics (30) Screening and Early Diagnosis in Pediatrics (30) Computer-Aided Diagnosis (135) mHealth for Diagnosis (63) Digital Health Reviews (1326)

Published on in Vol 26 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/57641, first published .
Accuracy of Machine Learning in Discriminating Kawasaki Disease and Other Febrile Illnesses: Systematic Review and Meta-Analysis

Accuracy of Machine Learning in Discriminating Kawasaki Disease and Other Febrile Illnesses: Systematic Review and Meta-Analysis

Accuracy of Machine Learning in Discriminating Kawasaki Disease and Other Febrile Illnesses: Systematic Review and Meta-Analysis

Authors of this article:

Jinpu Zhu1 Author Orcid Image ;   Fushuang Yang2 Author Orcid Image ;   Yang Wang3 Author Orcid Image ;   Zhongtian Wang1 Author Orcid Image ;   Yao Xiao1 Author Orcid Image ;   Lie Wang2 Author Orcid Image ;   Liping Sun2 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Review

    1College of Chinese Medicine, Changchun University of Chinese Medicine, Changchun, China

    2Center of Children's Clinic, The Affiliated Hospital to Changchun University of Chinese Medicine, Changchun, China

    3Beijing Jishuitan Hospital, Capital Medical University, Beijing, China

    *these authors contributed equally

    Corresponding Author:

    Liping Sun, MD

    Center of Children's Clinic

    The Affiliated Hospital to Changchun University of Chinese Medicine

    No. 185, Shenzhen Street

    Economic and Technological Development Zone, Jilin, P.R.C.

    Changchun, 130022

    China

    Phone: 86 15948000551

    Email: slpcczyydx@sina.com


    Background: Kawasaki disease (KD) is an acute pediatric vasculitis that can lead to coronary artery aneurysms and severe cardiovascular complications, often presenting with obvious fever in the early stages. In current clinical practice, distinguishing KD from other febrile illnesses remains a significant challenge. In recent years, some researchers have explored the potential of machine learning (ML) methods for the differential diagnosis of KD versus other febrile illnesses, as well as for predicting coronary artery lesions (CALs) in people with KD. However, there is still a lack of systematic evidence to validate their effectiveness. Therefore, we have conducted the first systematic review and meta-analysis to evaluate the accuracy of ML in differentiating KD from other febrile illnesses and in predicting CALs in people with KD, so as to provide evidence-based support for the application of ML in the diagnosis and treatment of KD.

    Objective: This study aimed to summarize the accuracy of ML in differentiating KD from other febrile illnesses and predicting CALs in people with KD.

    Methods: PubMed, Cochrane Library, Embase, and Web of Science were systematically searched until September 26, 2023. The risk of bias in the included original studies was appraised using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Stata (version 15.0; StataCorp) was used for the statistical analysis.

    Results: A total of 29 studies were incorporated. Of them, 20 used ML to differentiate KD from other febrile illnesses. These studies involved a total of 103,882 participants, including 12,541 people with KD. In the validation set, the pooled concordance index, sensitivity, and specificity were 0.898 (95% CI 0.874-0.922), 0.91 (95% CI 0.83-0.95), and 0.86 (95% CI 0.80-0.90), respectively. Meanwhile, 9 studies used ML for early prediction of the risk of CALs in children with KD. These studies involved a total of 6503 people with KD, of whom 986 had CALs. The pooled concordance index in the validation set was 0.787 (95% CI 0.738-0.835).

    Conclusions: The diagnostic and predictive factors used in the studies we included were primarily derived from common clinical data. The ML models constructed based on these clinical data demonstrated promising effectiveness in differentiating KD from other febrile illnesses and in predicting coronary artery lesions. Therefore, in future research, we can explore the use of ML methods to identify more efficient predictors and develop tools that can be applied on a broader scale for the differentiation of KD and the prediction of CALs.

    J Med Internet Res 2024;26:e57641

    doi:10.2196/57641

    Keywords

    machine learning; artificial intelligence; Kawasaki disease; febrile illness; coronary artery lesions; systematic review; meta-analysis 


    Background

    Kawasaki disease (KD) is a systemic vasculitis of small and medium-sized vessels, which has nonspecific symptoms and predominantly afflicts children aged 5 years or younger [1]. The incidence of KD worldwide varies greatly among different regions and ethnic groups, ranging from 46.3 to 359 cases per 100,000 people in East Asia, compared with 4.5 to 20 cases per 100,000 children aged 5 years or younger in Europe and North America [2]. KD is regarded as the most frequent cause of pediatric-acquired heart disease in developed countries. Its primary sequelae are related to the coronary system and may have a huge impact on the myocardium, valves, and arterial walls in some pediatric patients, which is less commonly affected in other febrile illnesses [3]. Early recognition of KD and timely administration of gammaglobulin significantly reduces the risk of coronary artery lesions (CALs), promotes the regression of coronary aneurysms, and decreases the likelihood of long-term health complications in affected children with KD [3-5].

    Currently, the etiology and exact causative agent of KD remain unknown. Its clinical diagnosis is made mainly on the basis of the patient’s clinical manifestations, including fever, skin and mucosal changes, swollen lymph nodes, extremity changes, rash, and cardiac manifestations, such as coronary artery dilatation or aneurysm [3,6]. Early and accurate recognition of KD is challenging due to several factors, its clinical symptoms do not appear in a specific sequence and may not manifest simultaneously; the characteristic symptoms and laboratory findings often overlap with those of other febrile illnesses, such as respiratory infections, sepsis, and multisystem inflammatory syndrome; and its self-limited nature in the acute phase can cause symptoms to diminish or disappear before a definitive diagnosis is made [3,7,8]. People with KD who do not respond adequately to initial intravenous immunoglobulin (IVIG) therapy are at a higher risk of developing CALs. Therefore, classical scoring systems for predicting IVIG nonresponsiveness, such as the Kobayashi score [9], Harada score [10], and Egami score [11], are also considered valuable for assessing the risk of CALs. However, the sensitivity of these established scoring systems outside Japan is not good. In addition, since infectious agents and immune regulatory dysfunction in the body may be the causative factors of KD, stress-related inflammatory markers occurring after exogenous infections, such as the erythrocyte sedimentation rate (ESR), C-reactive protein (CRP) level, white blood cell (WBC) count, and platelet (PLT) count, are thought to have the ability to predict CALs in people with KD, and cardiac markers including N-terminal pro-B-type natriuretic peptide and markers of cardiomyocyte damage are also considered useful for predicting CALs. However, the predictive performance of single predictors is limited [12-14]. Therefore, at present, how to efficiently diagnose KD and how to early predict CALs in people with KD remain urgent.

    Since Alan Turing proposed the theoretical foundation of artificial intelligence (AI) in the 1930s, the field has undergone over 70 years of development. Following the turn of the millennium, with the proliferation of computer technology and the internet, the surge in data availability and enhanced computational power have led to the widespread adoption of machine learning (ML) techniques [15]. Alongside advancements in medical testing technologies, vast amounts of medical imaging, laboratory monitoring data, and genetic sequencing results now require interpretation by physicians. The heavy load of medical reports and the complex communication processes between doctors and patients have contributed to diagnostic errors, inefficiencies in medical practice, and physician burnout. As a result, there is a growing clinical need for automated, intelligent, and reasonably accurate ML tools to assist in these tasks [16-18]. Classic ML techniques are broadly categorized into supervised and unsupervised learning. Compared with the latter, the former is more frequently used in the medical field because it can generate clinically applicable conclusions by learning from and analyzing large volumes of labeled data (such as laboratory data, medical images, and genetic data). This makes supervised learning particularly valuable for tasks such as disease prediction, diagnosis, and image segmentation in clinical practice [19]. ML can analyze complex data from a wide range of sources, such as clinical manifestations, laboratory results and imaging data. Unlike traditional approaches that focus on single biochemical indicators, ML can detect subtle correlations within large datasets, helping to address challenges that traditional statistical methods cannot solve. This capability enhances both predictive and diagnostic accuracy [20]. In the field of disease prediction, ML can forecast disease progression and adverse outcomes, which is crucial for developing targeted treatment and rehabilitation regimens. While, in disease diagnosis, ML is primarily applied to the diagnosis of complex diseases and the assessment of disease states. This can significantly aid in formulating specialized diagnostic strategies, thereby reducing the time and economic costs associated with diagnosis [21-24].

    Objective

    In recent years, some researchers have attempted to apply ML to differentiate KD from other febrile illnesses and to predict CALs in people with KD. However, there is still a lack of systematic evidence supporting the effectiveness of ML methods in these areas, which presents challenges for the development or refinement of AI diagnostic tools. Therefore, we have conducted the first systematic review and meta-analysis to evaluate the accuracy of ML in differentiating KD from other febrile illnesses and in predicting CALs in people with KD, so as to provide evidence-based support for the future development and application of AI in these fields.


    Study Registration

    This study was executed according to the operating guidelines for systematic reviews and meta-analyses and prospectively registered on PROSPERO (CRD42023481662).

    Eligibility Criteria

    The inclusion and exclusion criteria are shown in Textbox 1.

    Textbox 1. Inclusion and exclusion criteria.

    Inclusion criteria

    • To differentiate Kawasaki disease (KD) from other febrile illnesses, people with KD and people with febrile illnesses suspected to be KD were included as study participants. To predict coronary artery lesions (CALs) in KD, people with confirmed KD were included as study subjects.
    • The types of studies included were case-control studies, cohort studies, nested case-control studies, and case-cohort studies.
    • A predictive model for identifying KD or predicting CALs in people with KD was constructed entirely.
    • Some studies lacked an independent validation set and only adopted k-fold cross-validation or Bootstrap. Their contribution cannot be denied. Therefore, they were also included in our systematic review.
    • A small number of studies may be based on distinct machine learning (ML) models published on the same dataset, but they were also incorporated into our systematic review.
    • Included studies are published in English.

    Exclusion criteria

    • Meta-analyses, reviews, guidelines, expert opinions, and similar types of studies.
    • Only risk factor analysis was executed, and a full ML model was not constructed.
    • Studies that did not include any of the following outcome measures for assessing the predictive accuracy of ML models: Receiver Operating Characteristic, Concordance statistic, Concordance index, sensitivity, specificity, accuracy, recovery, precision, confusion matrix, fourfold table for estimating the quality of a diagnostic test, F1-score, and calibration curve.
    • Studies on the validation of mature scale.
    • Studies on the predictive accuracy of single-factor predictors.

    Data Sources and Search Strategy

    PubMed, Cochrane Library, Embase, and Web of Science were systematically searched until September 26, 2023. The search terms were designed based on a combination of subject headings and free-text words, without geographical restriction or time limit (year). The search strategy is detailed in Table S1 in Multimedia Appendix 1.

    Study Selection and Data Extraction

    The retrieved records were imported into Endnote, and duplicate publications were excluded. Then we read the titles and abstracts to screen out the original studies that did not meet the requirements. Next, full texts were downloaded and read to select eligible studies.

    A standard electronic data extraction spreadsheet was created before data extraction. The following data were extracted: titles of the original studies, the name of the first author, the year of publication, the author’s nationality, the study type, the data source, the number of people with KD, the total number of study participants, the numbers and total number of people with KD in the training set and validation set when a model was trained to identify KD, the numbers and total number of patients developing CALs in the training set and validation set when a model was trained for early prediction of CALs in KD, model validation methods, methods for handling missing values, methods for variable screening, types of models used, and predictors used in the final model.

    Literature screening and data extraction were carried out independently by 2 researchers (JZ and FY), and then, the results were cross-checked. Discrepancies were resolved by the third researcher (YW).

    Risk of Bias in Studies

    The risk of bias in eligible studies was appraised using the PROBAST (Prediction model Risk Of Bias Assessment Tool), which consisted of numerous questions across 4 different domains: participants, predictors, outcome, and analysis. This approach provides a comprehensive evaluation of the overall risk of bias and overall usability [25]. The 4 domains covered 2, 3, 6, and 9 specific questions, respectively. There are 3 answers (yes [Y]/probably yes [PY], no [N]/probably no [PN], and no information [NI]) to each question. If at least 1 question in a domain was answered as N or PN, it was rated at high risk. To be considered at low risk, all questions in the domain should be answered as Y or PY. The overall risk of bias was graded as low when all domains were considered at low risk, and the overall risk of bias was rated high risk when at least one domain was rated at high risk.

    Risk of bias assessment was conducted independently by 2 researchers [JZ and ZW] based on the PROBAST, and then, their results were cross-checked. Discrepancies were resolved by the third researcher [LS].

    Outcome Measures

    The primary outcome measures were the concordance index (C-index), sensitivity, and specificity reflecting the predictive accuracy of ML models. The secondary outcome measure was the frequency of use of a predictor to construct a ML model.

    Meta-Analysis Methods

    The meta-analysis was performed using Stata (version 15.0; Stata Corp). A meta-analysis was conducted of the measure (C-index) to assess the overall accuracy of ML models. If 95% CIs and standard errors of the C-index were missing in some original studies, its standard errors were estimated with reference to a study by Debray et al [26]. A random-effects model was leveraged for meta-analysis of the C-index given the differences in variables and inconsistent parameters included in various ML models.

    In addition, a bivariate mixed-effects model was leveraged for meta-analysis of sensitivity and specificity, which required that meta-analysis of sensitivity and specificity should be based on the fourfold table (2 × 2 table) for estimating the quality of a diagnostic test. However, the fourfold table was not reported in some original studies. Therefore, the fourfold table was calculated by the following 2 methods, calculation of sensitivity, specificity, and precision combined with the number of patients; and calculation of the best sensitivity and specificity determined based on the Youden index and then combined with the number of patients. When conducting a meta-analysis of the C-index, we used the “metan” program package, whereas for the meta-analysis of the fourfold table (2 × 2 table), we used the “midas” program package.


    Study Selection

    A total of 815 records were searched from the databases. Of them, 196 were removed due to duplication, and 569 were excluded after screening of titles and abstracts. The full texts of the remaining 50 articles were searched and read. Of them, 2 were excluded without peer review of abstracts, and 19 were excluded for the reasons described in Figure 1. In the end, a total of 29 studies were included [27-55].

    ‎
    Figure 1. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flow diagram for study selection.

    Study Characteristics

    This study included a total of 29 publications between 2001 and 2023, involving 29 ML models with variables all derived from relatively common clinical characteristics.

    Twenty studies involved a total of 20 models. They, published from 2013 to 2023, focused on the differential diagnosis of KD from Febrile controls (including respiratory infections, gastrointestinal infections, urinary infections, encephalitis, multisystem inflammatory syndrome, sepsis and cervical lymphadenopathy with fever due to bacterial infections, viral infections, or indeterminate pathogens). Of them, 11 were conducted in China [27-37], 5 in the United States [38-42], 1 in Egypt [43], 1 in Germany [44], 1 in Italy [45], and 1 in South Korea [46]. These studies involved a total of 103,882 study participants, including 12,541 people with KD.

    The types of models used in these 20 studies included logistic regression (9/20, 45%), support vector machine (3/20, 15%), least absolute shrinkage and selection operator (2/20, 10%), artificial neural network (2/20, 10%), extreme gradient boosting (1/20, 5%), decision tree (1/20, 5%), random forest (1/20, 5%), and linear discriminant analysis (1/20, 5%).

    The other 9 studies (involving 9 models) focused on the prediction of the risk of CALs in KD children. Of them, 5 were conducted in China [47-51], 2 in Japan [52,53], 1 in Spain [54], and 1 in the United States [55]. The types of models used in these 9 studies included multivariate logistic regression (8/9, 88.89%) and an artificial neural network (1/9, 11.11%). These 9 studies involved a total of 6503 people with KD, including 986 ones with CALs. Characteristics of including studies are presented in Table S2 and Table S3 in Multimedia Appendix 1.

    Risk of Bias in Studies

    In the differential diagnosis of KD from other febrile illnesses, 9 models [27-30,32,33,35,36,46] were based on data from retrospective studies, and therefore had a high risk of participant selection bias; 4 models [31,37,40,45] were constructed without data sources reported, so they were at unclear risk of bias. In the domain of predictors, 20 models were constructed without quality control measures reported for predictor assessment, so they were at an unclear risk of bias. In terms of outcomes, 20 models were all at low risk of bias. Regarding statistical analysis, 14 models included in 13 studies [27,29-31,33,35,37,40-43,45,46] were constructed with poor sample size design, missing data handled inappropriately or predictors screened irrationally, so they were at high risk of bias, as shown in Figure 2. The risk of bias in including studies is presented in Table S4 and Table S5 in Multimedia Appendix 1.

    ‎
    Figure 2. Risk assessment of machine learning models to identify Kawasaki disease from febrile controls.

    In the early prediction of CALs in people with KD, 5 models [48-52] were based on data from retrospective case-control studies, and therefore had a high risk of participant selection bias. In predictors and outcomes as domains, 9 models were all at low risk of bias. In statistical analysis as a domain, 6 models [47,51-55] were at high risk of bias because of irrational sample size design, and 3 models [48-50] were at high risk of bias owing to incorrect use of internal validation methods as shown in Figure 3.

    ‎
    Figure 3. Risk assessment of machine learning models to predict coronary artery lesions in people with Kawasaki disease.

    Meta-Analysis

    Differentiation of Kawasaki Disease From Other Febrile Illnesses

    In the training set, a random-effects model was leveraged for meta-analysis of the C-index, which was 0.898 (95% CI 0.874-0.922), as shown in Figure 4 [29-36,40-46]. The funnel plot showed no significant publication bias among the included studies. The results of meta-analysis for the 2 × 2 table showed a sensitivity of 0.89 (95% CI 0.83-0.92) and a specificity of 0.84 (95% CI 0.80-0.87) in the training set, as shown in Figure 5 [27,29-36,38,40-46].

    ‎
    Figure 4. Concordance index forest plots of machine learning to identify Kawasaki disease and febrile controls in training and validation sets [29-36,40-46].
    ‎
    Figure 5. Sensitivity and specificity of machine learning to identify Kawasaki disease and febrile controls in training sets [27,29,30-36,38,40-46].

    In the validation set, a random-effects model was used for meta-analysis of the C-index, which was 0.881 (95% CI 0.837-0.925), as shown in Figure 4 [29-36,40-46]. The funnel plot showed that there appeared to be publication bias among the included studies. The results of meta-analysis for the 2 × 2 table showed a sensitivity of 0.91 (95% CI 0.83-0.95) and a specificity of 0.86 (95% CI 0.80-0.90) in the validation set, as shown in Figure 6 [28,32-35,37-39,42,44,46].

    ‎
    Figure 6. Sensitivity and specificity of machine learning to identify Kawasaki disease and febrile controls in validation sets [28,32-35,37-39,42,44,46].
    Prediction of Coronary Artery Lesions in Kawasaki Disease

    In the training set, a random-effects model was used for meta-analysis of the C-index, which was 0.809 (95% CI 0.761-0.857), as shown in Figure 7 [47-55]. The funnel plot showed no significant publication bias among the included studies. The results of meta-analysis for the 2 × 2 table showed a sensitivity of 0.79 (95% CI 0.68-0.86) and a specificity of 0.83 (95% CI 0.65-0.93) in the training set, as shown in Figure 8 [48-54].

    ‎
    Figure 7. Concordance index forest plots of machine learning to predict coronary artery lesions in Kawasaki disease in training and validation sets [47-55].
    ‎
    Figure 8. Sensitivity and specificity of machine learning to predict coronary artery lesions in Kawasaki disease in training sets [48-54].

    In the validation set, a random-effects model was used for meta-analysis of the C-index, which was 0.787 (95% CI 0.738-0.835), as shown in Figure 7 [47-55]. The funnel plot showed that there appeared to be publication bias among the included studies. Since the number of models in the validation set was less than 4 and did not reach the lower limit for meta-analysis, we only described the ranges of sensitivity and specificity in the validation set, which were 0.220-0.952 and 0.644-0.755, respectively.

    Predictors

    Frequencies of Use of Diagnostic Factors to Construct ML Models for Differentiation of KD From Other Febrile Illnesses

    Regarding the use of diagnostic factors for differentiation of KD, there were a total of 39 diagnostic factors including CRP (n=11), neutrophils (NEUT; n=11), WBC (n=10), PLT (n=8), lymphocytes (n=8), hemoglobin (n=7), alanine aminotransferase (n=7), gamma-glutamyl transferase (n=7), eosinophils (n=7), monocytes (n=6), ESR (n=5), oral changes (n=5), conjunctival injection (n=5), extremity changes (n=4), age (n=3), cervical lymphadenopathy (n=3), rash (n=3), albumin (n=2), prognostic nutritional index (n=2), prealbumin (n=2), and duration (days) of fever (n=2). The following diagnostic factors were each used with a frequency of 1: Plasma Hepcidin, Pyuria, Urinary Leukocytes, Vomiting, Abdominal Pain, N-terminal Pro B-type Natriuretic Peptide, Thyroid Hormone Uptake, Height, complement C3, Procalcitonin, Neck Abscesses, Blood Phosphorus, Uric Acid, Chloride Compounds in serum, lactate dehydrogenase, aspartate aminotransferase, globulin, and temperature (Figure 9).

    ‎
    Figure 9. Frequencies of use of diagnostic factors to construct machine learning models to identify Kawasaki disease from febrile controls.
    Frequencies of Use of Predictors to Construct ML Models for Prediction of Coronary Artery Lesions in KD

    Regarding the use of diagnostic factors for prediction of CALs, there were a total of 26 predictors including CRP (n=7), IVIG resistance (n=4), duration of fever before IVIG infusion (n=3), age (n=3), baseline Z score (n=2), high-density Lipoprotein (n=2), albumin (n=2), PLT (n=2), ESR (n=2), duration of fever after IVIG infusion (n=1), monocytes (n=1), serum amyloid A (n=1), NEUT (n=1), lymphocytes (n=1), rash (n=1), oral changes (n=1), cervical lymphadenopathy (n=1), sex (n=1), race (n=1), creatinine (n=1), number of Steroid Pulse therapy sessions (n=1), Urinary β2-microglobulin (n=1), Hgb (n=1), Immunoglobulin A (n=1), immunoglobulin G (n=1), and matrix metalloproteinase 9 (n=1; Figure 10).

    ‎
    Figure 10. Frequencies of use of predictors to construct machine learning models to predict coronary artery lesions in Kawasaki disease.

    Principal Findings

    Our study included a total of 29 ML prediction models, of which 20 models focused on differentiating KD from other febrile illnesses, and 9 models focused on predicting the risk of CALs in children with KD. This study unraveled that ML achieved a favorable performance in differentiating KD from other febrile illnesses and predicting the risk of CALs in children with KD. In the validation set for ML to differentiate KD from other febrile illnesses, the C-index, sensitivity, and specificity were 0.881 (95% CI 0.837-0.925), 0.91 (95% CI 0.83-0.95), and 0.86 (95% CI 0.80-0.90), respectively. In the validation set for early prediction of the risk of CALs in children with KD, the C-index was 0.787 (95% CI 0.738-0.835), with a sensitivity range of 0.220-0.952 and a specificity range of 0.644-0.755. All the results in the validation set versus the training set showed no overfitting.

    CRP, NEUT, and WBC were the most frequently used diagnostic factors in the diagnostic model for KD, and CRP was also the most frequent predictor in the predictive model for CALs in people with KD. CRP is a very common laboratory test item. All the models we included were constructed based on interpretable ML models and routine clinical or laboratory variables, suggesting that desirable predictive results can be obtained if common clinical characteristics and laboratory variables are combined with appropriate ML modeling methods.

    Comparison With Previous Reviews

    In recent years, ML has been extensively applied in the diagnosis and early prediction of diseases to enhance precision and efficiency in health care, but the value of its application in the diagnosis of KD and early prediction of the risk of CALs in KD children is still unknown. A systematic review of existing studies is necessary to guide the development and application of ML in this field. To our knowledge, our study is the first systematic review evaluating the accuracy of ML for differentiation of KD from other febrile illnesses and for early prediction of CALs in children with KD.

    In clinical practice, there is an emphasis on using rapid and cost-effective diagnostic methods for children with fever and at risk of coronary artery disease. Most of the studies included in our review are based on common and routine clinical data. This indicated that when faced with patients with fever, clinicians can use ML methods and the most routine tests to differentiate and diagnose KD.

    Over the past few years, many teams have attempted to diagnose KD using genomic technology in addition to common clinical characteristics and laboratory variables. A meta-analysis by Zheng et al [56] showed that total mixed miRNAs detection for diagnosing KD had a sensitivity of 0.7 (0.66-0.74), a specificity of 0.87 (0.83-0.90), and an Summary Receiver Operating Characteristic of 0.8302. In contrast, our analysis showed that ML had a higher diagnostic accuracy. Another meta-analysis [57] showed that compared with ML, ncRNA had a similar diagnostic accuracy with an AUC of 0.90 (0.87-0.92) in differentiating KD from febrile illnesses. Some studies have found the same genetic association with risk for CAL in KD children. Xie et al [58] reviewed 164 previous studies and found that BTNL2, CASP3, FCGR2A, FGF23, FGβ, GRIN3A, HLA-E, IL10, ITPKC, and TGFBR2 were linked to a high risk for CALs in children with KD. Another meta-analysis [59] showed that Allele A at the functional SNP rs72689236 of the gene encoding caspase-3 was a genetic marker for susceptibility to KD with CALs. Although genomic technology has demonstrated some value and significance in the diagnosis of KD and the prediction of KD with CALs, genetic testing is time consuming and not applicable to the diagnosis of the acute phase of the disease. Considering the higher cost of genomic technology, the need for high-quality control in the laboratory and the complexity of genomic data analysis and interpretation, genomic technology is less clinically applicable to the early diagnosis of KD and the prediction of concomitant CALs. In contrast, common clinical variables are more applicable and convenient in most current clinical settings, while being more conducive to reducing the economic burden on people with KD.

    During the application of ML in the field of medicine, model type selection is critical to balance the accuracy and interpretability of a model. Interpretable models, such as logistic regression and the decision tree, tend to have a worse accuracy in identifying outcome measures. However, less interpretable models, such as Random Forest, XGboost, and Artificial Neural Networks, tend to have a higher accuracy. Logistic regression was the main ML model included in our study, suggesting that early diagnosis of KD and early prediction of CALs in people with KD were feasible based on routinely interpretable models.

    In addition, the selection of model variables is crucial for the construction of ML models, because it determines the testing accuracy of ML models. In current clinical practice, model variables can be classified into several types below, including common interpretable clinical characteristics, image-based radiomic features, and pathologic characteristics. Considering that it is difficult to avoid excessive imaging equipment configuration and bias in image segmentation during obtainment of variables through imaging and that pathologic characteristics have limitations including difficulty in quantification and limited data interpretation, we should prefer to use common clinical characteristics as model variables among these variables. The ML model variables included in our study were dominated by common interpretable clinical characteristics, which suggests that easy-to-use clinical tools can be developed in the future based on common clinical characteristics.

    Advantages and Limitations of the Study

    Our study is the first systematic review evaluating the accuracy of ML for the differentiation of KD from other febrile illnesses and for early prediction of CALs in children with KD. It was aimed at providing medical evidence for this field. However, our study has the following limitations. First, despite an extensive search, the number of original studies included is still limited, thereby resulting in a failure to conduct effective subgroup analysis based on ML model types in our study. Second, most of the ML models included in our review were constructed using random sampling for internal validation, with only a few studies using an independent external validation set to verify the accuracy of the models. This limitation somewhat restricts the generalizability and interpretation of our results. Third, we did not quantitatively describe the associations between the predictors (model variables) and KD or CALs in KD.

    Conclusions

    ML is desirably effective in differentiating KD from other febrile illnesses, and it is also able to fairly predict the occurrence of CALs. However, the development of ML models in the related field is still limited by a small amount of training data, its low quality and a single validation method. Thus, more subsequent multicenter studies are needed to provide more diverse data for ML models, enhance data quality, improve the validation method and outcome measures for ML models, refine the existing models, and develop new high-quality models. In future work, a quantitative characterization of the association between predictors (model variables) and both KD and CALs in KD is needed. In the meantime, more multicenter studies are desired to validate our conclusions. Furthermore, we plan to explore how to further validate and apply ML technologies in clinical practice.

    Acknowledgments

    This work was supported by the National Natural Science Foundation of China (82405483), the Department of Science and Technology of Jilin Province (20200603008SF; YDZJ202401673ZYTS), and the Jilin Provincial Administration of Traditional Chinese Medicine (2023129).

    Authors' Contributions

    All authors contributed to the study conception and design. JZ and YX wrote the original draft preparation. LW and LS conducted review and editing. JZ contributed to conceptualization. JZ, FY and ZW handled methodology. JZ, FY, YW, ZW, and LS managed formal analysis and investigation. LW and LS handled supervision. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. LW and LS contributed equally as the corresponding authors of this manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Additional tables.

    DOCX File , 48 KB
    Multimedia Appendix 2

    PRISMA checklist.

    DOCX File , 32 KB
    1. Agarwal S, Agrawal DK. Kawasaki disease: etiopathogenesis and novel treatment strategies. Expert Rev Clin Immunol. 2017;13(3):247-258. [FREE Full text] [CrossRef] [Medline]
    2. Watts RA, Hatemi G, Burns JC, Mohammad AJ. Global epidemiology of vasculitis. Nat Rev Rheumatol. 2022;18(1):22-34. [CrossRef] [Medline]
    3. McCrindle BW, Rowley AH, Newburger JW, Burns JC, Bolger AF, Gewitz M, et al. American Heart Association Rheumatic Fever, Endocarditis, and Kawasaki Disease Committee of the Council on Cardiovascular Disease in the Young; Council on Cardiovascular and Stroke Nursing; Council on Cardiovascular Surgery and Anesthesia; and Council on Epidemiology and Prevention. Diagnosis, treatment, and long-term management of Kawasaki disease: a scientific statement for health professionals from the American heart association. Circulation. 2017;135(17):e927-e999. [CrossRef] [Medline]
    4. Galeotti C, Bayry J, Kone-Paut I, Kaveri SV. Kawasaki disease: aetiopathogenesis and therapeutic utility of intravenous immunoglobulin. Autoimmun Rev. 2010;9(6):441-448. [CrossRef] [Medline]
    5. Gordon JB, Kahn AM, Burns JC. When children with kawasaki disease grow up: myocardial and vascular complications in adulthood. J Am Coll Cardiol. 2009;54(21):1911-1920. [FREE Full text] [CrossRef] [Medline]
    6. Ayusawa M, Sonobe T, Uemura S, Ogawa S, Nakamura Y, Kiyosawa N, et al. Kawasaki Disease Research Committee. Revision of diagnostic guidelines for Kawasaki disease (the 5th revised edition). Pediatr Int. 2005;47(2):232-234. [CrossRef] [Medline]
    7. Verdoni L, Mazza A, Gervasoni A, Martelli L, Ruggeri M, Ciuffreda M, et al. An outbreak of severe Kawasaki-like disease at the Italian epicentre of the SARS-CoV-2 epidemic: an observational cohort study. Lancet. 2020;395(10239):1771-1778. [FREE Full text] [CrossRef] [Medline]
    8. Godfred-Cato S, Bryant B, Leung J, Oster ME, Conklin L, Abrams J, et al. California MIS-C Response Team. COVID-19-associated multisystem inflammatory syndrome in children - United States, March-July 2020. MMWR Morb Mortal Wkly Rep. 2020;69(32):1074-1080. [CrossRef] [Medline]
    9. Kobayashi T, Inoue Y, Takeuchi K, Okada Y, Tamura K, Tomomasa T, et al. Prediction of intravenous immunoglobulin unresponsiveness in patients with Kawasaki disease. Circulation. 2006;113(22):2606-2612. [CrossRef] [Medline]
    10. Harada K. Intravenous gamma-globulin treatment in Kawasaki disease. Acta Paediatr Jpn. 1991;33(6):805-810. [CrossRef] [Medline]
    11. Egami K, Muta H, Ishii M, Suda K, Sugahara Y, Iemura M, et al. Prediction of resistance to intravenous immunoglobulin treatment in patients with Kawasaki disease. J Pediatr. 2006;149(2):237-240. [CrossRef] [Medline]
    12. de Zorzi A, Colan SD, Gauvreau K, Baker AL, Sundel RP, Newburger JW. Coronary artery dimensions may be misclassified as normal in Kawasaki disease. J Pediatr. 1998;133(2):254-258. [CrossRef] [Medline]
    13. Chaiyarak K, Durongpisitkul K, Atta T, Soongswang J, Laohaprasitiporn D, Nana A. Clinical manifestations of Kawasaki disease: what are the significant parameters? Asian Pac J Allergy Immunol. 2009;27(2-3):131-136. [FREE Full text] [Medline]
    14. Lin KH, Chang SS, Yu CW, Lin SC, Liu SC, Chao HY, et al. Usefulness of natriuretic peptide for the diagnosis of Kawasaki disease: a systematic review and meta-analysis. BMJ Open. 2015;5(4):e006703. [FREE Full text] [CrossRef] [Medline]
    15. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. [CrossRef] [Medline]
    16. Singh H, Giardina TD, Meyer AND, Forjuoh SN, Reis MD, Thomas EJ. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med. 2013;173(6):418-425. [FREE Full text] [CrossRef] [Medline]
    17. Kassam AF, Cortez AR, Winer L, Conzen K, El-Hinnawi A, Jones C, et al. Extinguishing burnout: national analysis of predictors and effects of burnout in abdominal transplant surgery fellows. Am J Transplant. 2021;21(1):307-313. [FREE Full text] [CrossRef] [Medline]
    18. Hodkinson A, Zhou A, Johnson J, Geraghty K, Riley R, Zhou A, et al. Associations of physician burnout with career engagement and quality of patient care: systematic review and meta-analysis. BMJ. 2022;378:e070442. [FREE Full text] [CrossRef] [Medline]
    19. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230-243. [FREE Full text] [CrossRef] [Medline]
    20. Bi Q, Goodman KE, Kaminsky J, Lessler J. What is machine learning? A primer for the epidemiologist. Am J Epidemiol. 2019;188(12):2222-2239. [CrossRef] [Medline]
    21. Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial intelligence in precision cardiovascular medicine. J Am Coll Cardiol. 2017;69(21):2657-2664. [FREE Full text] [CrossRef] [Medline]
    22. Dorado-Díaz PI, Sampedro-Gómez J, Vicente-Palacios V, Sánchez PL. Applications of artificial intelligence in cardiology. The future is already here. Rev Esp Cardiol (Engl Ed). 2019;72(12):1065-1075. [CrossRef] [Medline]
    23. Kim IB, Park SC. Machine learning-based definition of symptom clusters and selection of antidepressants for depressive syndrome. Diagnostics (Basel). 2021;11(9):1631. [FREE Full text] [CrossRef] [Medline]
    24. Shao J, Ma J, Zhang Q, Li W, Wang C. Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology. Semin Cancer Biol. 2023;91:1-15. [FREE Full text] [CrossRef] [Medline]
    25. Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST Group†. Probast: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51-58. [FREE Full text] [CrossRef] [Medline]
    26. Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, et al. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 2019;28(9):2768-2786. [FREE Full text] [CrossRef] [Medline]
    27. Yang YL, Kuo HC, Chen KD, Chu CH, Kuo K, Guo MM, et al. Combination of hemoglobin-for-age Z-score and plasma hepcidin identified as a novel predictor for Kawasaki disease. Children (Basel). Jun 18, 2022;9(6):93. [FREE Full text] [CrossRef] [Medline]
    28. Tsai CM, Lin CR, Kuo HC, Cheng FJ, Yu HR, Hung TC, et al. Use of machine learning to differentiate children with Kawasaki disease from other febrile children in a pediatric emergency department. JAMA Netw Open. 2023;6(4):e237489. [FREE Full text] [CrossRef] [Medline]
    29. Li C, Liu Y, Zhang D, Han Y, Chen B, Long Y, et al. A machine learning model for distinguishing Kawasaki disease from sepsis. Sci Rep. 2023;13(1):12553. [FREE Full text] [CrossRef] [Medline]
    30. Guo X, Liao J, Fan X, Xu M. Exploring the diagnostic value of eosinophil-to-lymphocyte ratio to differentiate Kawasaki disease from other febrile diseases based on clinical prediction model. Sci Rep. 2023;13(1):3399. [FREE Full text] [CrossRef] [Medline]
    31. Weng KP, Li SC, Chien KJ, Tsai KW, Kuo HC, Hsieh KS, et al. Prediction model for diagnosis of Kawasaki disease using iTRAQ-Based analysis. Children (Basel). 2021;8(7):576. [FREE Full text] [CrossRef] [Medline]
    32. Tsai CM, Chu CH, Liu X, Weng KP, Liu SF, Huang YH, et al. A novel score system of blood tests for differentiating Kawasaki disease from febrile children. PLoS One. 2021;16(1):e0244721. [FREE Full text] [CrossRef] [Medline]
    33. Huang YS, Liu XP, Xia HB, Cui LN, Lang XL, Liu CY, et al. Complement 3 and the prognostic nutritional index distinguish Kawasaki disease from other fever Illness with a nomogram. Children (Basel). 2021;8(9):825. [FREE Full text] [CrossRef] [Medline]
    34. Liu XP, Huang YS, Xia HB, Sun Y, Lang XL, Li QZ, et al. A nomogram model identifies eosinophilic frequencies to powerfully discriminate Kawasaki disease from febrile infections. Front Pediatr. 2020;8:559389. [FREE Full text] [CrossRef] [Medline]
    35. Liu XP, Huang YS, Kuo HC, Xia HB, Yi-Sun, Huang WD, et al. A novel nomogram model for differentiating Kawasaki disease from sepsis. Sci Rep. 2020;10(1):13745. [FREE Full text] [CrossRef] [Medline]
    36. Huang Z, Tan XH, Wang H, Pan B, Lv T, Tian J. A new diagnostic model to distinguish Kawasaki disease from other febrile illnesses in Chongqing: a retrospective study on 10,367 patients. Front Pediatr. 2020;8:533759. [FREE Full text] [CrossRef] [Medline]
    37. Huang J, Chen J. Bp neural network model for early diagnosis of Kawasaki disease. Advanced materials research. 2012;468:723-726. [FREE Full text] [CrossRef]
    38. Hao S, Jin B, Tan Z, Li Z, Ji J, Hu G, et al. Pediatric Emergency Medicine Kawasaki Disease Research Group. A classification tool for differentiation of Kawasaki disease from other febrile illnesses. J Pediatr. 2016;176:114-120.e8. [FREE Full text] [CrossRef] [Medline]
    39. Hao S, Ling XB, Kanegaye JT, Bainto E, Dominguez SR, Heizer H, et al. Pediatric Emergency Medicine Kawasaki Disease Research Group. Multicentre validation of a computer-based tool for differentiation of acute Kawasaki disease from clinically similar febrile illnesses. Arch Dis Child. 2020;105(8):772-777. [CrossRef] [Medline]
    40. Portman MA, Magaret CA, Barnes G, Peters C, Rao A, Rhyne R. An artificial intelligence derived blood test to diagnose Kawasaki disease. Hosp Pediatr. 2023;13(3):201-210. [CrossRef] [Medline]
    41. Lam JY, Shimizu C, Tremoulet AH, Bainto E, Roberts SC, Sivilay N, Pediatric Emergency Medicine Kawasaki Disease Research Group, et al. CHARMS Study Group. A machine-learning algorithm for diagnosis of multisystem inflammatory syndrome in children and Kawasaki disease in the USA: a retrospective model development and validation study. Lancet Digit Health. 2022;4(10):e717-e726. [FREE Full text] [CrossRef] [Medline]
    42. Ling XB, Kanegaye JT, Ji J, Peng S, Sato Y, Tremoulet A, et al. Point-of-care differentiation of Kawasaki disease from other febrile illnesses. J Pediatr. 2013;162(1):183-188.e3. [FREE Full text] [CrossRef] [Medline]
    43. Sobh A, Mosa DM, Khaled N, Korkor MS, Noureldin MA, Eita AM, et al. How multisystem inflammatory syndrome in children discriminated from Kawasaki disease: a differentiating score based on an inception cohort study. Clin Rheumatol. 2023;42(4):1151-1161. [FREE Full text] [CrossRef] [Medline]
    44. Du S, Mansmann U, Geisler BP, Li Y, Hornung R. A diagnostic model for Kawasaki disease based on immune cell characterization from blood samples. Front Pediatr. 2021;9:769937. [FREE Full text] [CrossRef] [Medline]
    45. Castaldo A, D'Anna C, Gelzo M, Giannattasio A, Maglione M, Muzzica S, et al. Immunophenotyping of peripheral blood cells allows to discriminate MIS-C and Kawasaki disease. Transl Med Commun. 2022;7(1):22. [FREE Full text] [CrossRef] [Medline]
    46. Kim JM, Kim J. Prediction model for the differential diagnosis of Kawasaki disease and acute cervical lymphadenitis in patients initially presenting with fever and cervical lymphadenitis. J Pediatr. 2020;225:30-36.e2. [CrossRef] [Medline]
    47. Yang C, Yang Y, Cao S, Ma Z, Du H, Li J, et al. Kawasaki disease coronary artery lesions prediction with monocyte-to-high-density lipoprotein ratio. Pediatr Res. 2023;94(1):246-251. [CrossRef] [Medline]
    48. Wu R, Jiang W, Sun Y, Wu L, Di Y, Wang J, et al. Indicators of oxidative stress in the prediction of coronary artery lesions in patients with Kawasaki disease. J Clin Rheumatol. 2023;29(3):126-131. [CrossRef] [Medline]
    49. Chen J, Li J, Yue Y, Liu Y, Xie T, Peng JQ, et al. Nomogram for predicting coronary artery lesions in patients with Kawasaki disease. Clin Cardiol. 2023;46(11):1434-1441. [FREE Full text] [CrossRef] [Medline]
    50. Yu H, Sun W, Wu H, Zhang S, Xu Z, Wu R, et al. The significance of symptoms in predicting coronary artery aneurysms of Kawasaki disease, especially in female patients. Front Cardiovasc Med. 2022;9:823862. [FREE Full text] [CrossRef] [Medline]
    51. Wang L, Yang Y, Cui Q, Cui Y, Li Q, Che X, et al. Evaluating the added predictive ability of MMP-9 in serum for Kawasaki disease with coronary artery lesions. J Investig Med. 2021;69(1):13-19. [CrossRef] [Medline]
    52. Azuma J, Yamamoto T, Nitta M, Hasegawa Y, Kijima E, Shimotsuji T, et al. Structure equation model and neural network analyses to predict coronary artery lesions in Kawasaki disease: a single-centre retrospective study. Sci Rep. 2020;10(1):11868. [FREE Full text] [CrossRef] [Medline]
    53. Morikawa Y, Ohashi Y, Harada K, Asai T, Okawa S, Nagashima M, et al. Coronary risks after high-dose gamma-globulin in children with Kawasaki disease. Pediatr Int. 2000;42(5):464-469. [CrossRef] [Medline]
    54. Grasa CD, Fernández-Cooke E, Domínguez-Rodríguez S, Aracil-Santos J, Barrios Tascon A, Sánchez-Manubens J, et al. Spanish group for the study of Kawasaki Disease (KAWA-RACE). Risk scores for Kawasaki disease, a management tool developed by the KAWA-RACE cohort. Clin Rheumatol. 2022;41(12):3759-3768. [CrossRef] [Medline]
    55. Son MBF, Gauvreau K, Tremoulet AH, Lo M, Baker AL, de Ferranti S, et al. Risk model development and validation for prediction of coronary artery aneurysms in Kawasaki disease in a North American population. J Am Heart Assoc. 2019;8(11):e011319. [FREE Full text] [CrossRef] [Medline]
    56. Zheng X, Li Y, Yue P, Ma F, Zhang Y, Wu G. Diagnostic significance of circulating miRNAs in Kawasaki disease in China: current evidence based on a meta-analysis. Medicine (Baltimore). 2021;100(6):e24174. [FREE Full text] [CrossRef] [Medline]
    57. Zhong X, Jia X, Wang H, Chen G, Li H, Li P, et al. Diagnostic significance of noncoding RNAs in kawasaki disease: a systematic review and meta-analysis. Front Pediatr. 2022;10:1071434. [FREE Full text] [CrossRef] [Medline]
    58. Xie X, Shi X, Liu M. The roles of genetic factors in Kawasaki disease: a systematic review and meta-analysis of genetic association studies. Pediatr Cardiol. 2018;39(2):207-225. [CrossRef] [Medline]
    59. Peng Q, Chen C, Wu Q, Yang Y. Association of new functional SNP rs72689236 of CASP3 with Kawasaki disease: a meta-analysis. Zhongguo Dang Dai Er Ke Za Zhi. 2013;15(6):477-483. [FREE Full text] [Medline]

    ‎
    AI: artificial intelligence
    C-index: concordance index
    CAL: coronary artery lesion
    CRP: C-reactive protein
    ESR: erythrocyte sedimentation rate
    IVIG: intravenous immunoglobulin
    KD: Kawasaki disease
    ML: machine learning
    NEUT: neutrophils
    PLT: platelet
    PROBAST: the Prediction Model Risk of Bias Assessment Tool
    WBC: white blood cell

    Edited by A Schwartz; submitted 22.02.24; peer-reviewed by Q Xia, N Goyal, N Goyal; comments to author 24.06.24; revised version received 25.07.24; accepted 20.09.24; published 18.11.24.

    Copyright

    ©Jinpu Zhu, Fushuang Yang, Yang Wang, Zhongtian Wang, Yao Xiao, Lie Wang, Liping Sun. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.