Original Paper
Abstract
Background: A meta-analysis is a quantitative, formal study design in epidemiology and clinical medicine that systematically integrates and quantitatively synthesizes findings from multiple independent studies. This approach not only enhances statistical power but also enables the exploration of effects across diverse populations and helps resolve controversies arising from conflicting studies.
Objective: This study aims to develop and implement a user-friendly tool for conducting meta-analyses, addressing the need for an accessible platform that simplifies the complex statistical procedures required for evidence synthesis while maintaining methodological rigor.
Methods: The platform available at MetaAnalysisOnline.com enables comprehensive meta-analyses through an intuitive web interface, requiring no programming expertise or command-line operations. The system accommodates diverse data types including binary (total and event numbers), continuous (mean and SD), and time-to-event data (hazard rates with CIs), while implementing both fixed-effect and random-effect models using established statistical approaches such as DerSimonian-Laird, Mantel-Haenszel, and inverse variance methods for effect size estimation and heterogeneity assessment.
Results: In addition to statistical tests, graphical representations including the forest plot, the funnel plot, and the z score plot can be drawn. A forest plot is highly effective in illustrating heterogeneity and pooled results. The risk of publication bias can be revealed by a funnel plot. A z score plot provides a visual assessment of whether more research is needed to establish a reliable conclusion. All the discussed models and visualization options are integrated into the registration-free web-based portal. Leveraging MetaAnalysisOnline.com's capabilities, we examined treatment-related adverse events in patients with cancer receiving perioperative anti–PD-1 immunotherapy through a systematic review encompassing 10 studies with 8099 total participants. Meta-analysis revealed that anti–PD-1 therapy doubled the risk of adverse events (risk ratio 2.15, 95% CI 1.39-3.32), with significant between-study heterogeneity (I2=95%) and publication bias detected through the Egger test (P=.02). While these findings suggest increased toxicity associated with anti–PD-1 treatment, the z score analysis indicated that additional studies are needed for definitive conclusions.
Conclusions: In summary, the web-based tool aims to bridge the void for clinical and life science researchers by offering a user-friendly alternative for the swift and reproducible meta-analysis of clinical and epidemiological trials.
doi:10.2196/64016
Keywords
Introduction
Systematic literature reviews in medicine summarize the best-proven knowledge from a research domain into a single paper []. A meta-analysis is the statistical combination of results from at least 2 separate studies which can serve as a powerful tool for summarizing, integrating, and interpreting quantitative research findings from the collected sources. The first potential advantage of a meta-analysis is its capacity to enhance precision. Many studies, when considered in isolation, may be too small to provide convincing evidence about intervention effects, and a meta-analysis of multiple studies can provide the necessary power to investigate whether interventions have an impact on the incidence of rare events.
Second, meta-analyses allow for the exploration of questions not explicitly addressed by individual studies. Primary studies often focus on specific participant types and interventions with well-defined parameters. By incorporating studies with diverse characteristics, it becomes possible to investigate the consistency of effects across a broader range of populations and interventions. Additionally, this approach may shed light on reasons for variations in effect estimates.
Third, meta-analyses can play a role in settling controversies that may arise from apparently conflicting studies or in generating new hypotheses. The statistical synthesis of findings enables formal assessment of conflicting results, providing a means to explore and quantify the reasons for different outcomes.
On the other hand, meta-analyses can be limited by inadequate search strategies, lack of clarity on comparators, insufficient assessment of risk of bias in primary studies, failure to address heterogeneity across studies, and the use of inappropriate or nonstandard methodological approaches, all of which can undermine the reliability and validity of the conclusions drawn []. Recent systematic evaluation of meta-analyses focusing on digital biomarker interventions revealed significant methodological inadequacies, with 92% of studies demonstrating critically low quality, highlighting the need for more rigorous methodological approaches in meta-analytic research []. Furthermore, meta-analyses often fail to adequately consider important demographic factors and subgroup analyses, as evidenced by a recent overview of systematic reviews on digital technologies in chronic obstructive pulmonary disease where only 1 out of 30 reviews included age-based subgroup analysis, despite the potential impact of such factors on treatment outcomes and clinical applicability [].
Here, the study aims to establish a new web portal for the simple and rapid execution of meta-analysis studies. In a meta-analysis, first, a summary statistic is computed for each study to describe the observed intervention effect uniformly across all studies. Then, a combined intervention effect estimate is calculated as a weighted average of the intervention effects estimated in the individual studies. We list options used by our web-based tool and for an all-inclusive description of the different approaches we suggest consulting statistical papers like the publications by Haidich [] and by Tawfik et al []. In the last part of our paper, we describe a sample application where we analyzed the correlation between immune checkpoint blockade and adverse events.
Methods
Web-Based Meta-Analysis Portal
Our main goal was to create a web-based application that allows users to perform a complete meta-analysis of clinical trials with minimal effort. For this, we established a user-friendly portal that is accessible in any major web browser. The web application is running on an Ubuntu server (Canonical Ltd) powered by Apache. The application can be used to perform meta-analyses using binary (using total and event numbers), continuous (using mean and SD data), and time-to-event data (using hazard ratio and CI data). The results of each study can be pasted from commonly used spreadsheet applications like Microsoft Excel.
The calculations and plots used by the web-based system are done using meta, RTSA, and metafor packages in R programming environments (version 4.2.2; R Foundation for Statistical Computing). The Shiny user interface was created by using the shinyjs, shinydashboard, and rhandsontable R packages. The portal generates a forest plot to summarize the results of the meta-analysis. A 2-tailed P value below .05 indicates significance and is displayed in the analysis results. MetaAnalysisOnline.com uses the DerSimonian-Laird estimator to estimate tau-square [].
In addition to the forest plot, a funnel plot is displayed to detect potential bias visually, and a z score plot is used for assessing the power achieved by the cumulative sample number. While there is commercial software available for performing a meta-analysis [], the portal we developed can be accessed completely free, without the need for registration or logging in.
The Implemented Statistical Methods
Experimental data are typically classified into 2 main categories: clinical data, which detail the effects of specific treatments on individual participants; and epidemiological data, which unveil patterns of disease or mortality in groups of participants exposed to either single agents or various substances. The most common classification in such studies uses categorical variables. A categorical variable is nonnumerical, relying on qualitative properties such as treatment arm, race, or gender, among others. Unlike numerical variables, categorical variables lack a specific ordering and assume values from a finite set of possibilities.
In MetaAnalysisOnline.com, the 2 study arms can be compared by either the fixed-effect model or the random-effect model. The fixed effect model operates on the assumption that a single true effect size underlies all studies in the meta-analysis, hence the term “fixed effect” or “common-effect” or sometimes “equal-effects.” Any variations in observed effects are attributed to sampling error. The random effects model posits that the true effect may vary across studies due to inherent differences or heterogeneity between them. As studies encompass diverse participant compositions and different interventions, individual studies may yield varying effect sizes. If an infinite number of studies were conducted, the effect estimates from all studies would conform to a normal distribution and the pooled estimate would then represent the mean or average effect. The assumption is that the effect sizes observed in conducted studies represent a random sample from the universe of all possible effect sizes, hence the term “random effects” []. The random effects approach for aggregating evidence from a series of experiments that compare 2 treatments was originally proposed by DerSimonian and Laird []. They integrated the heterogeneity of effects into the analysis, providing a comprehensive assessment of the overall treatment efficacy. The heterogeneity can be assessed using the I2 and 𝜏2 statistics, where the 𝜏2 represents the between-trial variance of the effect size. The square root of the 𝜏2 provides an estimate of the SD of the effects in the analyzed studies. High values of 𝜏2 indicate substantial heterogeneity among the effect sizes, suggesting that the true effects are not consistent across studies and may be influenced by other factors []. The I2 method, developed by Higgins and Thompson [], offers a percentage value representing the variability observed in the effect size, independent of sampling error. One of its advantages lies in the ease of interpreting its results: I2 percentages of 25% correspond to low, 50% to medium, and 75% to high levels of heterogeneity, respectively [].
There are 4 commonly used methods of meta-analysis for dichotomous outcomes: 4 fixed-effect methods (Mantel-Haenszel [], Peto, Bakbergenuly [], and inverse variance) and 1 random-effect method (DerSimonian and Laird inverse variance []). The Peto method is limited to combining odds ratios (ORs), while the other 3 methods can combine ORs, risk ratios, or risk differences. The Mantel-Haenszel method is a fixed effect meta-analysis method that uses a different weighting scheme depending on the type of effect measure being used (eg, risk ratio or OR) []. A relatively recent development is the Bakbergenuly sample-size-weighted estimator approach, which offers an alternative by using a weighted average where each study’s weight is determined by its effective sample size []. In the fixed effect model, the weights are calculated as wi=1/v1 where v1 represents the variance of the study. In the calculation, also known as the inverse variance method, studies with lower variances are weighted higher. In the random effect model, the calculation of weights is different, and the formula is 𝑤i=1/ (𝜏2+vi), where 𝜏2 represents the square of the variance of the distribution of true effect sizes. The inverse-variance method’s key advantage is its wide applicability, as it can be used for all effect measures and not only for OR, risk ratio, or risk difference. For its execution, we need to have the point estimate and the variance of the treatment effect in each study [].
Quantitative variables can be either discrete or continuous. Discrete variables are constrained to a finite number of values (such as whole numbers), whereas continuous variables can assume any value, including all possible values within a given range (extending to an infinite number of decimal places). Standard methods for meta-analysis of continuous data rely on the assumption that the outcomes follow a normal distribution within each intervention arm in each study. The 3 commonly used summary statistics for a meta-analysis of continuous data include the mean difference (MD), the standardized mean difference (SMD), and the ratio of means. The choice of summary statistics for continuous data primarily depends on whether studies report the outcome using the same scale (in which case MD is used) or different scales (in which case the SMD is typically used). In the MD approach, the SDs and sample sizes are used to calculate the weight assigned to each study and studies with smaller SDs are given relatively larger weights. In the SMD approach, the SDs are used to standardize the MDs to a single scale and are also used in the computation of study weights. When comparing with MD, interpreting the results can be challenging due to the need to categorize the SMD. Typically, we use the categories outlined by Cohen, which indicate that an SMD of approximately 0.2 corresponds to a small effect, around 0.5 signifies a moderate effect, and approximately 0.8 indicates a large effect []. In addition to MD and SMD, MetaAnalysisOnline.com also includes the ratio of means, a more recent solution where instead of using the difference between groups, the log-transformed ratio of the means of the groups is used in the analysis. The advantage of this approach is that the output of the result is easily interpretable similar to the odds or risk ratio [].
Demonstration: The Correlation Between Immune Checkpoint Blockade and Adverse Events
In recent years, the addition of immune checkpoint blockade using anti–PD-1 drugs, such as pembrolizumab, nivolumab, and cemiplimab, into cancer therapy has improved clinical outcomes. However, the safety of the immune checkpoint blockade needs further evaluation. As a demonstration of the functionalities of MetaAnalysisOnline.com, a set of previous studies was analyzed to assess how incorporating immune checkpoint blockade into perioperative cancer therapy affects treatment-related adverse events in patients treated with anti–PD-1 therapy.
Results
Graphical Representation of the Overall Effect: The Forest Plot
A comprehensive forest plot displays a set of the following information.
- The name of the original studies used in the meta-analysis. It is crucial to ensure the uniqueness of study names, particularly when an author or study name is listed more than once.
- The number of events and the total number of participants in each group of the study.
- A graph representing the relative risk and 95% CIs. Each square in the plot embodies the study results, centered on the point estimate, with a horizontal line indicating the 95% CI. The diamond represents the overall meta-analysis estimate, with the center signifying the pooled estimate and the prediction interval indicating the confidence limits. The CI depicts the range of intervention effects compatible with the study’s result, while the size of the block draws attention to the studies with larger weight, which dominates the calculation of the summary result presented by the diamond at the bottom.
- The risk ratio or hazard ratio and 95% CI for each individual study and the overall meta-analysis.
- The percentage of weight assigned to each study, particularly when presenting pooled results. Studies with higher sample numbers or those with narrower CIs receive higher weights.
- Finally, at the bottom of the plot, the heterogeneity and the overall effect values. A significant P value for heterogeneity indicates a substantial difference between studies. However, they heavily rely on the assumption of a normal distribution for the effects across studies and can present challenges when the number of studies is small, potentially leading to artificially wide or narrow intervals.
The features discussed in this section are marked in the plot generated by demo data in 
Figure 1. In addition, a screenshot of the analysis page with a generated forest plot is shown in 
Figure 2.
Graphical Representation of Heterogeneity: The Funnel Plot
The second analysis available in MetaAnalysisOnline.com is the funnel plot designed to detect publication bias. Whether the primary studies for pooling are observational studies or randomized controlled trials, heterogeneity between studies is probable. We show the components of a funnel plot used to assess study heterogeneity in 
Figure 3. Each dot on the plot represents an individual study. The x-axis represents the hazard ratio, or in other cases the log of OR, indicating the effect size of the treatment in each study. The y-axis represents the standard error, reflecting study precision, with larger studies having better precision (within the green area) and smaller studies having worse precision (within the orange area). In the plot, the vertical solid black or red line serves as the reference line corresponding to no effect or a hazard rate of 1 (or log OR=0). The overall effect and 95% CIs are depicted by the dashed black lines.
If the funnel plot suggests the presence of publication bias, the trim-and-fill method can be used to estimate the number of missing studies and impute their effect sizes. This allows the meta-analysis to be “trimmed” of asymmetric studies, and “filled” with the estimated missing studies to create a more symmetric funnel plot. The web-based platform automatically displays a button to generate trim-and-fill plots in case of a significant Egger test.
Trial Sequential Analysis and the z Score Plot
MetaAnalysisOnline.com performs a trial sequential analysis by drawing the z score plot to illustrate the reliability of the sample size. The y-axis of a z score plot depicts the cumulative z score and the x-axis the cumulative sample size. The cumulative z curve is built by sequentially adding studies based on chronological order. The plot depicts the relationship between the collective sample size and the achieved significance. When the ideal sample size is attained, a decisive conclusion regarding the examined condition can be made. The components of a z score plot are visualized in 
Figure 4.
At the end of the z curve lies the most recently added study, falling into one of the following zones: “favoring outcome A,” “favoring outcome B,” “no correlation,” or “statistically not significant.” The first 2 zones indicate statistically significant results, which may vary until the optimal sample size is reached, but after this, a reliable final conclusion can be determined. Placement in the “no correlation” signifies strong evidence suggesting that subsequent studies are unlikely to alter the no-effect outcome significantly. Placement within the “not statistically significant” zone indicates the definitive requirement for further studies as neither correlation nor missing correlation could be determined.
Independent Validation
To verify the output of the digital portal, a set of 10 samples was drawn from the dataset of a simulated randomized trial developed for the metafor package and a meta-analysis was performed. Analysis was repeated using IBM SPSS (version 29). The following outputs were compared: OR, the calculated weight, overall effect, as well as measures of heterogeneity (I2 and τ²). For all parameters, the corresponding 95% CIs were also checked. The output results of the 2 methods were identical, indicating sufficient reliability of the implemented procedures.
Demonstration: Immune Checkpoint Blockade and Adverse Events
To show the entire analysis pipeline, the association between immune checkpoint blockade using anti–PD-1 agents and adverse events was analyzed. Altogether 10 studies were included with a total of 4379 participants in the anti–PD-1–treated cohort and 3720 participants in the control cohort [-]. Based on the analysis performed using the random effects model with the Mantel-Haenszel method to compare the risk ratio, there is a statistical difference between the 2 cohorts, the overall risk ratio is 2.15 with a 95% CI of 1.39-3.32. The test for overall effect reached significance as depicted in 
Figure 5A.
Notably, substantial heterogeneity was detected, suggesting inconsistent effects in magnitude and/or direction. The I2 value indicates that 95% of the variability among studies stems from heterogeneity rather than random chance.
The funnel plot indicates a potential publication bias. The Egger test supports the presence of funnel plot asymmetry (intercept: 3.85, 95% CI 1.39-6.3; t8=3.072; P=.02; visualized in Figure 5B).
Based on the z score plot, the total number of samples did not reach the optimal number, and more research is needed to establish a conclusion (Figure 5C).
Discussion
Here, we have established a new aid for those performing systematic reviews and meta-analyses. A major advantage of our tool is its direct design for life sciences researchers and medical professionals who generally lack extensive statistical and bioinformatic skills. The user-friendly interface enables the execution of the analysis in real-time. Entering the data is possible by directly copying from a database with a single click. The tool generates the 3 most important graphical representations–a forest plot, a funnel plot, and a z score plot—in real-time. The most important advantage of the tool over other software lies in its extremely user-friendly setup—neither installation nor registration is necessary for the use and the input page was set up to enable prompt copying of data from popular spreadsheet applications.
The most important graphical result provided by MetaAnalysisOnline.com is a forest plot, which serves as a visual depiction of both the outcomes from individual studies and the overall results of the analysis []. Additionally, it illustrates overall effect estimates and the heterogeneity of studies, reflecting the variation in results among individual studies []. The plot also includes the prediction interval, which is widely used to express the amount of heterogeneity in a meta-analysis []. Notably, many reviews in the literature use slightly different forest plots as there is no uniform standardization on their appearance. A previous study investigating over 2000 forest plots from the literature identified highly standardized plots in Cochrane reviews but missing key elements in non-Cochrane reviews. Certain formats were not optimal for information exchange, and a significant number of plots contained insufficient data to be considered useful [].
The second implemented visual result is the funnel plot. In this, results from smaller studies will scatter broadly at the bottom of the graph, while the dispersion is likely to decrease among larger studies. In the absence of bias, the plot will take on the appearance of a symmetrical, inverted funnel, with asymmetry suggesting the presence of a risk of publication bias []. A significant Egger test result means that the points on the funnel plot are not symmetric. It is important to acknowledge that in practical scenarios, the source of heterogeneity cannot be determined and is often unknown. It is important to distinguish between various types of heterogeneity. Clinical heterogeneity refers to variability in participants, interventions, and outcomes studied, while methodological heterogeneity pertains to variability in study design, outcome measurement tools, and risk of bias. Statistical heterogeneity arises from variability in the intervention effects being evaluated in different studies, which can be a result of clinical or methodological diversity, or both. I2 and 𝜏2 statistics can be used to assess study heterogeneity [] and although the funnel plot is primarily used to detect publication bias, it can also be used to visualize the heterogeneity []. To provide a meaningful summary, it is crucial to ensure that the group of studies is sufficiently homogeneous in terms of participants, interventions, and outcomes before conducting a meta-analysis.
A fundamental question omitted in many performed meta-analyses is assessing the robustness of the sample number reached []. In particular, meta-analyses face the risk of yielding misleading outcomes due to significant results that may be false positives (type I errors; α) or falsely insignificant findings (type II errors; β). These errors can stem from various sources such as low-quality or underpowered trials, publication bias, and excessive significance testing. To address this issue, trial sequential analysis (TSA) has been devised as a cumulative meta-analysis technique []. TSA aims to account for both type I and type II errors, providing a means to estimate the point at which the observed effect size becomes robust enough to resist further influence from additional studies. In other words, TSA offers guidance on the necessity of further studies and helps clinicians prevent unnecessary trials [].
Our tool was established to incorporate a simple way to conduct subgroup analysis as well—to examine the potential variability of treatment effects among different subgroups of patients or trials []. Subgroup analysis involves dividing all participant data in the meta-analysis into subsets based on patient characteristics (eg, age) or trial characteristics (eg, location), and then performing a meta-analysis on one or more of these subsets. These analyses can help to estimate treatment effects for clinically relevant subgroups of patients or to identify sources of heterogeneity []. The decision to conduct such analyses may be based on prior research suggesting that treatment effects could differ among different patient subgroups.
Two considerations should be kept in mind when running the statistical analysis using MetaAnalysisOnline.com. Sometimes the data points scatter over a wide range and truncating the data is advisable (to constrain the length of the follow-up time, the applied treatment dose, or any other descriptive characteristics of the participants) at a selected threshold in case of scarcity of data above the threshold value. A second issue refers to the inherent differences between the studies due to the varying number of included participants. To achieve the most robust results, studies should be weighed for sample size to better reflect the characteristics of the general population.
We demonstrate the use of MetaAnalysisOnline.com by performing a meta-analysis of studies examining the association between immune checkpoint blockade for anti-PD-1 agents and adverse events []. Although immune checkpoint blockade significantly increased the rate of adverse events, we observed a significant heterogeneity, and we can conclude that more research is needed to establish an accurate correlation.
Despite the meta-analysis’s aim to identify and evaluate all relevant studies meeting inclusion criteria, this goal may not always be fully realized. Some studies might be overlooked, particularly if they are not published in English or if they report nonsignificant results, making them less likely to be published. Understanding and accounting for these factors is essential for a comprehensive and critical appraisal of a meta-analysis.
Finally, we have to discuss some limitations of our tool. In order to retain user-friendliness, we had to make some compromises. The maximal number of studies in 1 analysis is limited to 100. We have established a default format for the plots, and the graphical parameters (eg, font type and size, spacing, and line thickness) cannot be changed. In addition, we used Shiny to enable seamless visualization and scaling. Because of this, the server behind the platform can sometimes slow down when a large number of studies are analyzed in 1 setting.
In summary, MetaAnlysisOnline.com can be used for a quick and comprehensive meta-analysis in clinical and epidemiological trials. When presenting the results of a meta-analysis, 3 key aspects need to be considered. First, we have to examine the pooled result, representing the overall combined outcome obtained by pooling the individual studies in a forest plot. While providing a synthesized perspective on the collective findings, we have to consider heterogeneity, which relates to variations in results, methodology, or study populations across the included studies. Second, we must assess the potential presence of the risk of publication bias. Finally, the z score plot can be used to determine whether the cumulative sample number is sufficient for a definitive conclusion, or whether additional studies are still needed.
Acknowledgments
The 4.0 version of ChatGPT, developed by OpenAI, was used as a language tool to refine our writing and enhance the clarity of our work. This project was supported by the National Research, Development, and Innovation Office (PharmaLab, RRF-2.3.1-21-2022-00015). A5 Genetics Ltd (Kutaso, Hungary) provided computational infrastructure for the study.
Data Availability
All data generated or analyzed during this study are included in this published article.
Authors' Contributions
FJT contributed to the study by developing the conceptual framework and designing the methodology. He curated and analyzed the data, ensuring the accuracy and validity of the findings. His expertise in software development and visualization allowed for the effective presentation of results. Throughout the research process, FJT played a key role in validating the data and refining the study through critical review and editing. BG provided the overall direction of the project, securing funding and managing resources to support its execution. He played a central role in conceptualizing the study, overseeing methodological development, and ensuring the validity of the findings. BG took the lead in drafting the original manuscript and also contributed to data visualization.
Conflicts of Interest
None declared.
References
- Paul J, Barari M. Meta‐analysis and traditional systematic literature reviews—What, why, when, where, and how? Psychol Mark. 2022;39(6):1099-1115. [CrossRef]
- Kajal F. Clarity on the type of review. Comment on "Value Cocreation in Health Care: Systematic Review". J Med Internet Res. 2022;24(7):e38457. [FREE Full text] [CrossRef] [Medline]
- Motahari-Nezhad H, Al-Abdulkarim H, Fgaier M, Abid MM, Péntek M, Gulácsi L, et al. Digital biomarker-based interventions: systematic review of systematic reviews. J Med Internet Res. 2022;24(12):e41042. [FREE Full text] [CrossRef] [Medline]
- Matthias K, Honekamp I, Heinrich M, De Santis KK. Consideration of sex, gender, or age on outcomes of digital technologies for treatment and monitoring of chronic obstructive pulmonary disease: overview of systematic reviews. J Med Internet Res. 2023;25:e49639. [FREE Full text] [CrossRef] [Medline]
- Haidich AB. Meta-analysis in medical research. Hippokratia. 2010;14:29-37. [FREE Full text] [Medline]
- Tawfik GM, Dila KAS, Mohamed MYF, Tam DNH, Kien ND, Ahmed AM, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47:46. [FREE Full text] [CrossRef] [Medline]
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188. [FREE Full text] [CrossRef] [Medline]
- Borenstein M. Comprehensive meta-analysis software. In: Systematic Reviews in Health Research. Hoboken, NJ. Wiley; 2022:535-548.
- Dettori JR, Norvell DC, Chapman JR. Fixed-effect vs random-effects models for meta-analysis: 3 points to consider. Global Spine J. 2022;12(7):1624-1626. [FREE Full text] [CrossRef] [Medline]
- Ruppar T. Meta-analysis: How to quantify and explain heterogeneity? Eur J Cardiovasc Nurs. 2020;19(7):646-652. [CrossRef] [Medline]
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558. [CrossRef] [Medline]
- Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods. 2006;11(2):193-206. [CrossRef] [Medline]
- Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst. 1959;22(4):719-748. [Medline]
- Bakbergenuly I, Hoaglin DC, Kulinskaya E. Methods for estimating between-study variance and overall effect in meta-analysis of odds ratios. Res Synth Methods. 2020;11(3):426-442. [CrossRef] [Medline]
- Takeshima N, Sozu T, Tajika A, Ogawa Y, Hayasaka Y, Furukawa TA. Which is more generalizable, powerful and interpretable in meta-analyses, mean difference or standardized mean difference? BMC Med Res Methodol. 2014;14:30. [FREE Full text] [CrossRef] [Medline]
- Friedrich JO, Adhikari NKJ, Beyene J. The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Med Res Methodol. 2008;8:32. [FREE Full text] [CrossRef] [Medline]
- Bajorin DF, Witjes JA, Gschwend JE, Schenker M, Valderrama BP, Tomita Y, et al. Adjuvant nivolumab versus placebo in muscle-invasive urothelial carcinoma. N Engl J Med. 2021;384(22):2102-2114. [FREE Full text] [CrossRef] [Medline]
- Eggermont AMM, Blank CU, Mandala M, Long GV, Atkinson VG, Dalle S, et al. Longer follow-up confirms recurrence-free survival benefit of adjuvant pembrolizumab in high-risk stage III melanoma: updated results from the EORTC 1325-MG/KEYNOTE-054 trial. J Clin Oncol. 2020;38(33):3925-3936. [FREE Full text] [CrossRef] [Medline]
- Choueiri TK, Tomczak P, Park SH, Venugopal B, Ferguson T, Chang YH, et al. Adjuvant pembrolizumab after nephrectomy in renal-cell carcinoma. N Engl J Med. 2021;385(8):683-694. [FREE Full text] [CrossRef] [Medline]
- Forde PM, Spicer J, Lu S, Provencio M, Mitsudomi T, Awad MM, et al. Neoadjuvant nivolumab plus chemotherapy in resectable lung cancer. N Engl J Med. 2022;386(21):1973-1985. [FREE Full text] [CrossRef] [Medline]
- Kelly RJ, Ajani JA, Kuzdzal J, Zander T, Van Cutsem E, Piessen G, et al. Adjuvant nivolumab in resected esophageal or gastroesophageal junction cancer. N Engl J Med. 2021;384(13):1191-1203. [CrossRef] [Medline]
- Long GV, Luke JJ, Khattak MA, de la Cruz Merino L, Del Vecchio M, Rutkowski P, et al. Pembrolizumab versus placebo as adjuvant therapy in resected stage IIB or IIC melanoma (KEYNOTE-716): distant metastasis-free survival results of a multicentre, double-blind, randomised, phase 3 trial. Lancet Oncol. 2022;23(11):1378-1388. [CrossRef] [Medline]
- O'Brien M, Paz-Ares L, Marreaud S, Dafni U, Oselin K, Havel L, et al. Pembrolizumab versus placebo as adjuvant therapy for completely resected stage IB-IIIA non-small-cell lung cancer (PEARLS/KEYNOTE-091): an interim analysis of a randomised, triple-blind, phase 3 trial. Lancet Oncol. 2022;23(10):1274-1286. [CrossRef] [Medline]
- Rahma OE, Yothers G, Hong TS, Russell MM, You YN, Parker W, et al. Use of total neoadjuvant therapy for locally advanced rectal cancer: initial results from the pembrolizumab arm of a phase 2 randomized clinical trial. JAMA Oncol. 2021;7(8):1225-1230. [FREE Full text] [CrossRef] [Medline]
- Schmid P, Cortes J, Dent R, Pusztai L, McArthur H, Kümmel S, et al. Event-free survival with pembrolizumab in early triple-negative breast cancer. N Engl J Med. 2022;386(6):556-567. [CrossRef] [Medline]
- Wakelee H, Liberman M, Kato T, Tsuboi M, Lee SH, Gao S, et al. Perioperative pembrolizumab for early-stage non-small-cell lung cancer. N Engl J Med. 2023;389(6):491-503. [FREE Full text] [CrossRef] [Medline]
- Lewis S, Clarke M. Forest plots: trying to see the wood and the trees. BMJ. 2001;322(7300):1479-1480. [FREE Full text] [CrossRef] [Medline]
- Dettori JR, Norvell DC, Chapman JR. Seeing the forest by looking at the trees: how to interpret a meta-analysis forest plot. Global Spine J. 2021;11(4):614-616. [FREE Full text] [CrossRef] [Medline]
- Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. [CrossRef] [Medline]
- Schriger DL, Altman DG, Vetter JA, Heafner T, Moher D. Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice. Int J Epidemiol. 2010;39(2):421-429. [CrossRef] [Medline]
- Afonso J, Ramirez-Campillo R, Clemente FM, Büttner FC, Andrade R. The perils of misinterpreting and misusing "Publication Bias" in meta-analyses: an education review on funnel plot-based methods. Sports Med. 2024;54(2):257-269. [FREE Full text] [CrossRef] [Medline]
- Thorlund K, Imberger G, Johnston BC, Walsh M, Awad T, Thabane L, et al. Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses. PLoS One. 2012;7(7):e39471. [FREE Full text] [CrossRef] [Medline]
- Sterne JAC, Harbord RM. Funnel plots in meta-analysis. Stata J. 2004;4(2):127-141. [CrossRef]
- Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JPA, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? Int J Epidemiol. 2009;38(1):276-286. [CrossRef] [Medline]
- Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol. 2008;61(1):64-75. [CrossRef] [Medline]
- Kang H. Trial sequential analysis: novel approach for meta-analysis. Anesth Pain Med. 2021;16(2):138-150. [FREE Full text] [CrossRef] [Medline]
- Chang Y, Phillips MR, Guymer RH, Thabane L, Bhandari M, Chaudhary V, et al. R.E.T.I.N.A. study group. The 5 min meta-analysis: understanding how to read and interpret a forest plot. Eye. 2022;36(4):673-675. [FREE Full text] [CrossRef] [Medline]
- Richardson M, Garner P, Donegan S. Interpretation of subgroup analyses in systematic reviews: a tutorial. Clin Epidemiol Glob Health. 2019;7(2):192-198. [CrossRef]
- Fujiwara Y, Horita N, Adib E, Zhou S, Nassar AH, Asad ZUA, et al. Treatment-related adverse events, including fatal toxicities, in patients with solid tumours receiving neoadjuvant and adjuvant immune checkpoint blockade: a systematic review and meta-analysis of randomised controlled trials. Lancet Oncol. 2024;25(1):62-75. [CrossRef] [Medline]
Abbreviations
| MD: mean difference |
| OR: odds ratio |
| SMD: standardized mean difference |
| TSA: trial sequential analysis |
Edited by X Ma; submitted 05.07.24; peer-reviewed by P Wu, H Nguyen; comments to author 14.10.24; revised version received 30.10.24; accepted 18.01.25; published 06.03.25.
Copyright©János Tibor Fekete, Balázs Győrffy. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.03.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.