Meta-Analyses as Bias Prone As Industry-Backed Trials

Clinical trials supported by the pharmaceutical industry are more likely to report positive outcomes than those backed by other sources, and the same pattern appears to hold for meta-analyses, concluded a review in the October 14 British Medical Journal.

The analysis by Anders Jørgensen, M.D., and Peter Gøtzsche of the Nordic Cochrane Center in Denmark, and Jørgen Hilden, an associate professor of biostatistics at the University of Copenhagen, compared pairs of meta-analyses or reviews that studied the same two drugs used to treat the same disorder.

One set of these reviews was produced by the Cochrane Collaboration, an independent international organization that produces and disseminates systematic reviews of health care interventions. Twenty-four Cochrane reviews were matched with meta-analysis papers published within two years before or after the online Cochrane review and with no author in common. The paper reviews were divided into those supported by industry, those with undeclared support, and those with nonprofit or no support. Support from the pharmaceutical industry meant grants, authorship, or major assistance such as help with statistical analysis.

The authors assessed the methodological quality of the reviews and examined their conclusions and recommendations.

They found no significant differences when comparing seven Cochrane reviews paired with those having nonprofit or no outside support, although three pairs of reviews came to different conclusions about the drugs being evaluated. Eight industry-supported reviews, however, “should be read with caution,” they said.

“The estimated treatment effects in industry-supported reviews were similar to those of Cochrane reviews, but the former had uniformly positive recommendations for the experimental drug, without reservations about methodological limitations of the trials or costs, in contrast to none of the Cochrane reviews,” wrote Jørgensen, Hilden, and Gøtzsche.“ This suggests that the main problem with industry-supported reviews lies in how conclusions are formulated.”

The Cochrane reviews more often avoided bias in selecting studies, described methods of concealing the patients' allocation to a treatment arm, described patients who were excluded from a study, used appropriate criteria for assessing validity, and used more sources to come up with studies to include in the review.

Reviews with undeclared financial support “were often biased and poorly done,” said the authors. Such studies produced results similar to industry-supported reviews but were more cautious in their conclusions.

The authors included several pairs of reviews of psychotropic drugs in their analysis. For instance, a pairing of antidepressant studies found that the industry-supported review failed to include 12 trials that appeared in the Cochrane review yet did include 20 unpublished trials with “data on file.” An industry-backed review comparing risperidone and haloperidol had problems with sample sizes and patient selection.

The Cochrane Collaboration forbids industry support of Cochrane reviews, although a pharmaceutical company funded secondary analyses of a review author's own work was included in a review before that policy was established.

The authors noted that they could not be blinded as to the source of the articles and that they are all affiliated with the Cochrane Collaboration. However, they provided details on their working methods to allow readers to form their own judgments, a practice they believe should extend to all systematic reviews or meta-analyses.

“Cochrane Reviews Compared With Industry-Supported Meta-Analyses and Other Meta-Analyses of the Same Drugs: Systematic Review” is posted at<http://bmj.bmjjournals.com/cgi/content/full/333/7572/782>.▪