Objective:

The broad objective of this report is to compare and contrast via simulation five meta-regression approaches that model the heterogeneity among study treatment effects: fixed effects with and without covariates; random effects with and without covariates; and control rate meta-regression.

Methodology:

We conducted a systematic review of MEDLINE^®, HealthSTAR, EMBASE, MANTIS, SciSearch^®, Social SciSearch^®, Allied and Complementary Medicine, the Current Index to Statistics, and the Methodology Register of the Cochrane Library from the inception of each database through March 2001 using the search terms “metaregress-” or “meta” within two words of “regress-.” We supplemented these searches with articles identified by experts, and by searching the reference lists of all relevant articles found. We constructed a statistical notation generally applicable to different meta-regression methods. We convened a one-day panel of nine experts, and elicited their recommendations for the practice of meta-regression and implementation of our simulation study. We implemented a large-scale simulation to compare and contrast the five meta-regression techniques.

Main Results:

We identified and categorized 85 publications on meta-regression. We presented scenarios for which meta-regression might be informative, and expressed the most common meta-regression models in a common notation. Our expert panel made several recommendations regarding the simulation parameters. The panel also identified the need for outreach by the methodological community to the user community in advising how to conduct, interpret, and present meta-regression analyses, including the development of software and diagnostic aids to assess models.

The simulation was a complete factorial design including all possible 7,776 combinations of the simulation parameters. The results were evaluated using an analysis-of-variance (ANOVA) model relating the simulation parameters to the bias in the estimation of the additive treatment effect. Across the five different meta-regression methods, six terms in a three-way ANOVA model were found to be practically important as they captured contributions to the bias of 10% or greater on average.

Conclusions:

Our simulation results produced specific guidelines for meta-regression practitioners that may be summarized in the key message that the causes of heterogeneity should be explored via the inclusion of covariates at both the person level and study level. Based on our comparison of bias across approaches, either fixed effects or random effects methods can be used to support this exploration. In terms of future simulation research, we need to increase the variability in sample sizes, explore correlations between study outcome rates and covariates at both the study and person level, and evaluate within-study variation for person-level covariate(s). We now have in place a simulation methodology, a common notation, and a supportive panel of national experts to enable and guide our continued work in this area. The research presented in this report has already impacted the application of meta-regression in several alternative medicine settings, and improved our ability to synthesize and understand these therapies.