Impact of Big Data Analytics on People’s Health: Overview of Systematic Reviews and Recommendations for Future Studies

Introduction

Big data analytics tools handle complex datasets that traditional data processing systems cannot efficiently and economically store, manage, or process. Through the application of artificial intelligence (AI) algorithms and machine learning (ML), big data analytics has potential to revolutionize health care, supporting clinicians, providers, and policymakers for planning or implementing interventions [1], faster disease detection, therapeutic decision support, outcome prediction, and increased personalized medicine, resulting in lower-cost, higher-quality care with better outcomes [1,2].

In 2018, the World Health Organization (WHO) proposed the expedited 13th General Programme of Work (GPW13), which was approved and adopted by its 194 Member States, focusing on measurable impacts on people’s health at the state level to transform public health with three core features: enhanced universal health coverage, health emergencies protection, and better health and well-being [3]. Forty-six outcome target indicators emerged from the GPW13, covering a range of health issues [3]. Big data analytics may help to support health policy decision-making, accelerate the achievement of the GPW13 core priorities and targets, and guide the roadmap for the European region based on the European Programme of Work (EPW) 2020/2025 [4,5].

Therefore, the aim of this study was to provide an overview of systematic reviews that assessed the effects of the use of big data analytics on people’s health according to the WHO core features defined in the GPW13 and the EPW. We included complex reviews that assessed multiple interventions, different populations, and differing outcomes resulting from big data analytics on people’s health, and identified the challenges, opportunities, and best practices for future research.

Methods

Study Design

This study was designed to provide an overview of systematic reviews in accordance with guidelines from the Cochrane Handbook for Systematic Reviews of Interventions, along with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the QUOROM (Quality of Reporting of Meta-analyses) guidelines [6-8]. The study protocol is published on PROSPERO (CRD42020214048).

Search Strategy

To identify records assessing the effect of big data analytics on people’s health, aligned with the WHO health indicators defined in the GPW13 (Textbox 1), a comprehensive and systematic search was performed using six multidisciplinary databases from their inception to September 21, 2020. The search strategy was designed in collaboration with a senior librarian and is described in detail in Multimedia Appendix 1.

References were imported into reference management software (EndNote X9) and duplicates were removed. Unique records were uploaded onto the Covidence Platform (Veritas Health Innovation) for screening, data extraction, and quality assessment. A manual search of reference lists was performed to supplement the search.

Study Selection

Peer-reviewed publications categorized as systematic reviews assessing the effects of big data analytics on any of the GPW13 and EPW health indicators and core priorities were included, regardless of language and study design. We only considered studies in which the search was performed in at least two databases, and included a description of the search strategy and the methodology used for study selection and data extraction. We only included studies that evaluated concrete relationships between the use of big data analytics and its effect on people’s lives, according to the WHO strategic priorities and indicators. Along with the 46 indicators listed in Textbox 1, we also included studies evaluating the use of big data during the COVID-19 pandemic. To identify gaps, we included reviews focusing on challenges, best practices, and short- and long-term opportunities related to big data technologies. Nonsystematic reviews, primary studies, opinions, short communications, nonscientific articles, conference abstracts, and reviews with big data inappropriately defined were excluded.

Although big data analysis is capable of handling large volumes of data, rather than focusing on the data volume/size, we focused on the process that defines big data analytics, which includes the following phases [9]: (1) data selection, (2) data preprocessing, (3) data transformation, (4) AI/expert systems, and (5) understanding/assessment. The first three phases include subtasks such as: (i) feature selection and extraction, (ii) data cleaning, and (iii) data integration from multiple sources. The included studies covered all phases of the process. Title, abstract, and full-text screening were independently performed by two reviewers using the inclusion criteria. Any disagreements were resolved by a third independent investigator.

Data Extraction

The following data were extracted from the retrieved articles: publication information, journal name and impact factor, study characteristics, big data characteristics, outcomes, lessons and barriers for implementation, and main limitations. Data were individually extracted by team members and cross-checked for accuracy by a second investigator.

Assessment of Methodological Quality of Included Reviews

Two researchers independently assessed the studies using the AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews 2) checklist, which includes the following critical domains, assessed in 16 items: protocol registered prior to review, adequacy of literature search, justification for excluded studies, risk of bias in included studies, appropriateness of meta-analytic methods, consideration of bias risk when interpreting results, and assessing the presence and likely impact of publication bias [10]. Appropriateness to each appraisal feature was rated as yes, no, partial yes, not applicable, or unclear. Any conflict was resolved by a third party. Studies with a review protocol tracking number were analyzed. A final summary score was given to each included record, rated as “critically low,” “low,” “moderate,” or “high” [10].

Data Synthesis

Results are reported in summary tables and through a narrative synthesis, grouping studies assessing the same disease or condition, and identifying challenges and opportunities. We also schematically represent the evidence and gaps from these reviews as an overall synthesis.

Results

Reviews Retrieved

The search retrieved 1536 publications, 112 of which were duplicates. Most of the studies were excluded after title and abstract analysis (n=1237), leaving 185 selected for full-text screening, and 35 [11-45] were ultimately included in the final analysis after applying the eligibility criteria according to the QUOROM guidelines [8] (Figure 1). Reference list screening did not retrieve any additional review. One study under “awaiting classification” could not be retrieved.

‎View this figure

Quality of Evidence in Individual Systematic Reviews

Multimedia Appendix 2 shows the detailed results of the quality assessment of the 35 systematic reviews. Overall, most of the reviews (n=25) were rated with “critically low” confidence in the results using the AMSTAR 2 criteria, with 7 rated “low” and 3 rated as “moderate.” None of the reviews achieved a “high” rating. Common methodological drawbacks included omission of prospective protocol submission or publication, inappropriate search strategy, lack of independent and dual literature screening and data extraction, absence of explanation for heterogeneity among the studies, unclear or no reasons for study exclusion, and lack of risk of bias assessment.

No standard critical appraisal tools were mentioned. Among the 12 reviews that performed any quality assessment, the Quality Assessment of Diagnostic Accuracy Studies 2 tool was used in four reviews demonstrating an overall low risk of bias [14,16,27,28], whereas other tools assessed the risk of bias in studies not specifically aiming at diagnostic accuracy features. El Idrissi et al [18] used their own quality assessment tool and Luo et al [34] used an adapted version of the Critical Appraisal Skills Programme. Appraisal of the quality of evidence aligned with the Grading of Recommendations Assessment, Development and Evaluation method was reported in only one review [17]. Many reviews did not evaluate bias.

Characteristics of Included Reviews

Summary features and main findings of the 35 systematic reviews are presented in Multimedia Appendix 3 and Multimedia Appendix 4, respectively. The included reviews were published in 34 different journals from 2007 to 2020. Most were published in English in a first-quartile journal with an impact factor ranging from 0.977 to 17.679. They covered over 2501 primary studies, involving at least 5,000,000 individuals. Only three reviews included meta-analyses, and one included a randomized clinical trial; the others were based on cohort studies.

Data Sources and Purposes of Included Studies

Many reviews included data collected from electronic medical records, hospital information systems, or any databank that used individual patient data to create predictive models or evaluate collective patterns [12,13,16-21,24-27,30,33-35,37,38,40,42-45]. Additionally, four reviews included primary studies based on imaging datasets and databanks, assessing different parameters of accuracy [15,29,31,36]. Other reviews focused on genetic databases [28,35], data from assisted reproductive technologies [30], or publicly available data [11,14,22,32]. Four studies lacked precision about the origin of the datasets used in their analysis or did not specifically use patient data in the investigation [23,37,39,41].

The purposes of the reviews varied broadly. Generally, they (1) outlined AI applications in different medical specialties; (2) analyzed features for the detection, prediction, or diagnosis of multiple diseases or conditions; or (3) pinpointed challenges and opportunities.

WHO Indicators and Core Priorities

Most of the studies assessed the effects of big data analytics on noncommunicable diseases [12-15,17,21,22,24,27,31,32,34,36, 38,40-44]. Furthermore, three reviews covered mental health, associated with the indicator “suicide mortality rate” [19,25,45]; three studies were related to the indicator “probability of dying from any of cardiovascular, cancer, diabetes, or chronic renal disease” [16,18,20,28,29]; and two studies were related to the indicator “proportion of bloodstream infections due to antimicrobial-resistant organisms” [26,33]. One study described technology use in disaster management and preparedness, covering the “number of persons affected by disasters” indicator [11], and one study was associated with the indicator “maternal mortality ratio” [30]. Overlap made precise classification into WHO health indicators challenging, and four studies could not be categorized because they mainly described challenges or opportunities in big data analytics [23,39] or because they were related to the COVID-19 pandemic [35,37].

Diseases or Conditions Assessed

Diabetes Mellitus

AI tools associated with big data analytics in the care of patients with diabetes mellitus (DM) were assessed in six reviews that included 345 primary studies [15,20,32,38,40]. Three studies reviewed AI in screening and diagnosing type 1 or type 2 DM, providing varied ranges of accuracy, sensitivity, and specificity [20,32,40]. Variables included systolic blood pressure, body mass index, triglyceride levels, and others. Two reviews covered DM control and the clinical management of DM patients [32,40]. One noted that techniques for diabetes self-management varied among the tools evaluated and reported mean values for its robust metrics [18]. The other evaluated the use of data-driven tools for predicting blood glucose dynamics and the impact of ML and data mining [20], describing the input parameters used among data-driven analysis models. However, the authors of these reviews concluded that achieving a methodologically precise predictive model is challenging and must consider multiple parameters.

Various studies assessed the ability of big data analytics to predict individual DM complications such as hypoglycemia, nephropathy, and others [15,32,38]. Supervised ML methods, decision trees, deep neural networks, random forests (RF) learning, and support vector machine (SVM) reportedly had the best outcomes for assessing complications. One review assessed deep learning–based algorithms in screening patients for diabetic retinopathy. Of 11 studies, 8 reported sensitivity and specificity of 80.3% to 100% and 84%% to 99%, respectively; two reported accuracies of 78.7% and 81%; and one reported an area under the receiver operating curve (AUC) of 0.955 [15].

Mental Health

Five reviews reported on AI, data mining, and ML in psychiatry/psychology [12,14,19,25,45], most commonly assessing these techniques in the diagnosis of mental disorders. Two reviews assessed the use of ML algorithms for predicting suicidal behaviors. High levels of risk classification accuracy (typically higher than 90%) were reported in two reviews, either for adult primary care patients or teenagers [19,25]. Although the review authors stated the potential of ML techniques in daily clinical practice, limitations were highlighted, including no external validation and reporting inconsistencies.

The use of ML algorithms for early detection of psychiatric conditions was also reported [12,45]. ML was used to develop prediagnosis algorithms for constructing risk models to signal a patient’s predisposition or risk for a psychiatric/psychological health issue, for predicting a diagnosis of newly identified patients, and to differentiate mental conditions with overlapping symptomatology. For studies using structural neuroimaging to classify bipolar diseases and other diagnoses, the accuracy ranged from 52.13% to 100%, whereas studies using serum biomarkers reported an accuracy ranging from 72.5% to 77.5%.

Only one review used social media to generate analyzable data on the prevention, recognition, and support for severe mental illnesses [14]. The study included broad descriptions of ML techniques and data types for detection, diagnosis, prognosis, treatment, support, and resulting public health implications. The authors highlighted the potential for monitoring well-being, and providing an ecologically and cost-efficient evaluation of community mental health through social media and electronic records.

COVID-19

Two reviews reported the application of big data analytics and ML to better understand the current novel coronavirus pandemic [35,37]. One assessed data mining and ML techniques in diagnosing COVID-19 cases. Although the study did not define the best methodology to evaluate and detect potential cases, the authors noted an elevated frequency of decision tree models, naïve Bayes classifiers, and SVM algorithms used during previous pandemics.

Another review focused on SARS-CoV-2 immunization, and proposed that AI could expedite vaccine discovery through studying the virus’s capabilities, virulence, and genome using genetic databanks. That study merged discussions of deep learning–based drug screening for predicting the interaction between protein and ligands, and using imaging results linked to AI tools for detecting SARS-CoV-2 infections.

Oncology

Four studies described the utility of ML, computerized clinical decision systems, and deep learning in oncology [24,28,29,31]. Using computerized clinical decision support systems (DSS) significantly improves process outcomes in oncology [24]. A compelling example shows that initial decisions were modified in 31% of cases after consultation of clinical DSS, which consistently resulted in improved patient management. Furthermore, implementing clinical DSS led to an average cost reduction of US $17,000 for lung cancer patients. A remarkable workload decrease reportedly occurs when these systems are implemented in oncology facilities, leading to improved patient management and adherence to guidelines [24].

One study evaluated ML techniques in a genomic study of head and neck cancers, and found a wide range of accuracy rates (56.7% to 99.4%) based on the use of genomic data in prognostic prediction. Lastly, two studies reported accuracy levels ranging from 68% to 99.6% when using deep learning algorithms in the automatic detection of pulmonary nodules in computerized tomography images.

Cardiovascular and Neurological Conditions

Six studies described the effect of big data analytics in cardiology [13,16,21,42] and neurology [43,44]. One review assessed the use of ML techniques for predicting cardiac arrest [42]. Different variables were used as predictors among individual studies, including electrocardiographic parameters, heart rate variability, echocardiography, and others. Supervised ML techniques were most frequently applied to predict cardiac arrest events, with clear evidence of regression techniques and SVM algorithms. The authors reported a mean AUC of 0.76 for risk score development and efficiency evaluation [42].

Similarly, two studies assessed the use of intelligent systems in diagnosing acute coronary syndrome and heart failure [13,21], demonstrating high accuracy levels using several methods such as SVM, feature selection, and neural networks. These studies also described useful clinical features for creating prediction and diagnostic models, such as patient clinical data, electrocardiogram characteristics, and cardiac biomarkers.

Scores to identify patients at higher risk to develop QT-interval prolongation have been developed, and predictive analytics incorporated into clinical decision support tools have been tested for their ability to alert physicians of individuals who are at risk of or have QT-interval prolongation [16].

Regarding stroke, two systematic reviews evaluated using ML models for predicting outcomes and diagnosing cerebral ischemic events [43,44]. Generally, ML models were most frequently associated with mortality prediction, functional outcomes, neurological deterioration, and quality of life. The diagnosis of ischemic stroke was associated with similar or better comparative accuracy for detecting large vessel occlusion compared with humans, depending on the AI algorithm employed [44]. RF algorithms had 68% sensitivity and over 80% specificity compared with humans. Analyses of convolutional neural network (CNN) algorithms were limited, but systems using CNNs reported performance metrics on average 8% to 10% greater than those of ML employing RF, with up to 85% mean sensitivity for automatic large vessel occlusion detection. However, AI algorithm performance metrics used different standards, precluding objective comparison. Core and perfusion studies from RAPID-computed tomography and magnetic resonance imaging had the highest metrics for AI accuracy, above 80%, with some datasets showing 100% sensitivity to predict favorable perfusion mismatch. The authors noted several errors of AI use in diagnosing stroke [44].

Miscellaneous Conditions

Several studies reported significant improvement in disease diagnosis and event prediction using big data analytics tools, including remarkable enhancement of sepsis prediction using ML techniques [26]. Another review provided moderate evidence that ML models can reach high performance standards in detecting health care–associated infections [33].

One review focused on the diagnostic accuracy of AI systems in analyzing radiographic images for pulmonary tuberculosis, mostly referring to development instead of clinical evaluation [27]. In studies assessing accuracy, the sensitivity ranged from 56% to 97%, specificity ranged from 36% to 95%, and the AUC ranged from 78% to 99%.

One review also assessed multiple sclerosis diagnosis. Among detection methodologies, rule-based and natural language processing methods were deemed to have superior diagnostic performance based of elevated accuracy and positive predictive value [41]. This study indicates that these methods have potential impacts for early recognition of the disease, increasing quality of life, and allowing prompt pharmacological and nonpharmacological intervention.

Asthma exacerbation events and predictive models for early detection were evaluated in one review, which reached a pooled diagnostic ability of 77% (95% CI 73%-80%) [17]. Among the included studies, most models for predicting asthma development had less than 80% accuracy. None of the 42 studies modeled the reincidence of exacerbation events, and overall accuracy performance was considered inadequate. However, the authors encouraged creating models using large datasets to increase prediction accuracy levels. Logistic regression and Cox proportional hazard regression appeared to be the most commonly used methodologies. Gastric tissue disease and the usability of deep learning techniques were evaluated in one study [36]. CNN was the most common model used for gastric problem classification or detection. Additionally, residual neural network and fully convolutional network were considered to be appropriate models for disease generation, classification, and segmentation.

Two reviews analyzed the use of big data analytics and AI in public health [22,30]. One listed the impact of continuous pharmacological exposure of pregnant women, emphasizing that AI could improve popular understanding of drug effects on pregnancy, mainly through: (i) reliable clinical information disclosure, (ii) adequate scientific research design, and (iii) implementation of DSS [30]. Another review assessing the use of big data in disaster preparedness evidenced that most existing methods are qualitative, covering the response phase of the disaster chain of events [11]. The utilized tools included data originating from geographic information systems, social media interfaces, and disaster prediction modeling studies.

Challenges and Opportunities

Two systematic reviews provided narrative evaluations of the challenges of big data analytics in health care [23,39]. Evidence from these two systematic reviews, and those from the other reviews, are summarized in Textbox 2.

Discussion

This overview is the first to assess the effects of big data analytics on the prioritized WHO indicators, which offers utility for noncommunicable diseases and the ongoing COVID-19 pandemic. Although the research question focused on the impact of big data analytics on people’s health, studies assessing the impact on clinical outcomes are still scarce. Most of the reviews assessed performance values using big data tools and ML techniques, and demonstrated their applications in medical practice. Most of the reviews were associated with the GPW13 indicator “probability of dying from any cardiovascular disease, cancer, diabetes, chronic respiratory disease.” This indicator outranks others because of the incidence, prevalence, premature mortality, and economic impact of these diseases [46]. Similarly, many reviews were related to “people requiring interventions against noncommunicable diseases.” The included reviews in this study addressed many necessary health-related tasks; however, the quality of evidence was found to be low to moderate, and studies assessing the impact on clinical outcomes are notably scarce.

The low to moderate quality of evidence suggests that big data analytics has moderate to high accuracy for the (1) diagnosis and prediction of complications of DM, (2) diagnosis of mental diseases, (3) prediction of suicidal behaviors, and (4) diagnosis of chronic diseases. Most studies presented performance values, although no study assessed whether big data analytics or ML could improve the early detection of specific diseases.

Clinical research and clinical trials significantly contribute to understanding the patterns and characteristics of diseases, as well as for improving detection of acute or chronic pathologies and to guide the development of novel medical interventions [47]. However, experimental/theoretical investigations, mathematical approaches, and computer-based studies hinge on handling sample size limitations and performing data imputation [48,49]. Computer-driven analysis can easily handle missing data, examine variable mechanisms in complex systems, and employ essential tools for exploratory evaluations using voluminous input data. Big data analytics can execute an operation on/process data within microseconds after generation of the dataset, allowing for real-time follow up [50,51]. These studies and prospective applications could generate innovative knowledge and promote actionable insights; however, adapting, validating, and translating scientific data into practical medical protocols or evaluation studies is necessary.

Many systematic reviews reported simple or inappropriate evaluation measures for the task at hand. The most common metric used to evaluate the performance of a classification predictive model is accuracy, which is calculated as the proportion of correct predictions in the test set divided by the total number of predictions that were made on the test set. This metric is easy to use and to interpret, as a single number summarizes the model capability. However, accuracy values and error rate, which is simply the complement of accuracy, are not adequate for skewed or imbalanced classification tasks (ie, when the distribution of observations in the training dataset across the classes is not equal), because of the bias toward the majority class. When the distribution is slightly skewed, accuracy can still be a useful metric; however, when the distribution is severely skewed, accuracy becomes an unreliable measure of model performance.

For instance, in a binary classification task with a distribution of (95%, 5%) for the classes (eg, healthy vs sick), a “dumb classifier” that simply chooses the class “healthy” for all instances will have 95% of accuracy in this task, although the most important issue in this task would be correctly classifying the “sick” class. Precision (also called the positive predictive value), which captures the fraction of correctly classified instances among the instances predicted for a given class (eg, “sick”); recall or sensitivity, which captures the fraction of instances of a class (eg, “sick”) that were correctly classified; and F-measure, the harmonic mean of precision and recall calculated per class of interest, are more robust metrics for several practical situations. The proper choice of an evaluation metric should be carefully determined, as these indices ought to be used by regulatory bodies for screening tests and not for diagnostic reasoning [52]. The most important issue is to choose the appropriate (most robust) performance metric given the particularities of each case.

Another pitfall identified among the included reviews was the lack of reporting the precise experimental protocols used for testing ML algorithms and the specific type of replication performed.

There is no formal tool for assessing quality and risk of bias in big data studies. This is an area that is ripe for development. In Textbox 3, we summarize our recommendations for systematic reviews on the application of big data and ML for people’s health based on our experience, the findings of this systematic review, and inspired by Cunha et al [53].

High variability in the results was evident across different ML techniques and approaches among the 35 reviews, even for those assessing the same disease or condition. Indeed, designing big data analysis and ML experiments involves elevated model complexity and commonly requires testing of several modeling algorithms [54]. The diversity of big data tools and ML algorithms requires proper standardization of protocols and comparative approaches. Additionally, the process of tuning the hyperparameters of the algorithms is not uniformly reported. Important characteristics essential for replicability and external validation were not frequently available. Lastly, most of the studies provide little guidance to explain the results. Without knowing how and why the models achieve their results, applicability and trust of the models in real-world scenarios are severely compromised. Therefore, we urge the testing and assessment of supervised, unsupervised, and semisupervised methodologies, with explanation and interpretation to justify the results. Moreover, we encourage hyperparameter optimization to achieve adjusted improvement of models, enhance model generalizations for untrained data, and avoid overfitting to increase predictive accuracy.

Only two published systematic reviews evaluated the impact of big data analytics on the COVID-19 pandemic. Primary studies on COVID-19 are lacking, which indicates an opportunity to apply big data and ML to this and future epidemics/pandemics [35,37]. As of November 30, 2020, many published protocols were retrieved through a standard search on PROSPERO. The titles of these review protocols showed an intention to evaluate ML tools in diagnosis and prediction, the impact of telemedicine using ML techniques, and the use of AI-based disease surveillance [55].

Although DSS are an important application of big data analytics and may benefit patient care [56-58], only two reviews assessed such systems [16,24]. One focused on predictive analytics for identifying patients at risk of drug-induced QTc interval prolongation, discussing the efficacy of a DSS that has shown evidence of reduced prescriptions for QT interval–prolonging drugs. Similarly, one study exploring the impact of DSS on quality care in oncology showed that implementing these systems might positively impact physician-prescribing behaviors, health care costs, and clinician workload.

This overview of systematic reviews updates the available evidence from multiple primary studies intersecting computer science, engineering, medicine, and public health. We used a comprehensive search strategy (performed by an information specialist) with a predefined published protocol, precise inclusion criteria, rigorous data extraction, and quality assessment of retrieved records. We avoided reporting bias through the dual and blinded examination of systematic reviews and by having one review author standardizing the extracted data.

However, limitations exist. The inferior quality scores based on the AMSTAR 2 tool might reflect incomplete reporting and lack of adherence to substandardized review methods. There is neither an established bias risk tool specifically for big data or ML studies nor any systematic way of presenting the findings of such studies. Furthermore, most studies provided a narrative description of results, requiring summarization. Nevertheless, all of the reviews were inspected by most authors, and the most relevant data were condensed in the text or in descriptive tables.

Big data analytics provide public health and health care with powerful instruments to gather and analyze large volumes of heterogeneous data. Although research in this field has been growing exponentially in the last decade, the overall quality of evidence is found to be low to moderate. High variability of results was observed across different ML techniques and approaches, even for the same disease or condition. The diversity of big data tools and ML algorithms require proper standardization of protocols and comparative approaches, and the process of tuning the hyperparameters of the algorithms is not uniformly reported. Important characteristics essential for replicability and external validation were not frequently available.

Additionally, the included reviews in this systematic review addressed different health-related tasks; however, studies assessing the impact on clinical outcomes remain scarce. Thus, evidence of applicability in daily medical practice is still needed. Further studies should focus on how big data analytics impact clinical outcomes and on creating proper methodological guidelines for reporting big data/ML studies, as well as using robust performance metrics to assess accuracy.