The Problem of Insufficient Reporting and Publication Bias

In October scientists from Germany’s Institute for Quality and Efficiency in Health Care (IQWiG) published a paper in the British Medical Journal (BMJ) analyzing clinical trial data on placebo, selective serotonin reuptake inhibitors (SSRIs), and Pfizer’s antidepressant reboxetine.  The object was to assess the benefits and harm of reboxetine versus placebo or SSRIs in the acute treatment of depression and to measure the impact of potential publication bias for reboxetine trials.

Dirk Eyding, who led the study, and his colleagues concluded that Pfizer’s reboxetine turns out to be “overall an ineffective and potentially harmful antidepressant.” Beate Wieseler, Ph.D., deputy head of IQWiG's department of drug assessment, added, “It is not a major drug in depression, but every patient that is treated with an ineffective drug is one too many.”

Reboxetine, indicated for the treatment of clinical depression, panic disorder, and ADD/ADHD, was originally developed by Pharmacia, which is now part of Pfizer. It sells under many trade names including Davedax, Edronax, Norebox, Prolift, Solvex, and Vestra. The drug acts by inhibiting reuptake of the neurotransmitter norepinephrine unlike the most common antidepressants such as Celexa (citalopram), Paxil (paroxetine), Prozac (fluoxetine), and Zoloft (sertraline), all SSRIs.

The treatment has been available in Europe for 13 years and is licensed in over 50 countries. In 1999, FDA granted preliminary approval on the basis of trials primarily conducted outside the U.S. However, after results of later U.S. and Canadian trials were submitted, the agency declined final approval in 2001 citing “a lack of compelling evidence of efficacy.”

Comprehensive Databases Needed

The IQWiG authors didn’t set out on a reboxetine witch hunt. But the investigators became alarmed when their analysis of 13 acute treatment trials that included 4,098 patients revealed that data on 74% of them were never published. The studies were placebo-controlled, SSRI-controlled, or both.
“Our comparison of published and unpublished trials confirmed this assumption: The positive benefit-risk ratio of reboxetine in the published literature was changed to a negative ratio if unpublished trials were added to the analysis,” the authors wrote. 

Unfortunately this is a tiresome, sleazy, and potentially dangerous old story. In August 2004, GlaxoSmithKline (GSK) agreed to pay $2.5 million to settle a suit alleging that the company had suppressed studies showing that antidepressant Paxil was no better than a placebo in treating depression in children. The case also stated that GSK hid results showing that Paxil might increase suicidal thoughts in children and teenagers.

At a House hearing in September 2004, Republican and Democratic lawmakers chastised pharma executives from several companies for hiding study results. Around the same time Merck and Pfizer were criticized for failing to disclose, until that year, clinical trial results that indicated that cox-2 painkillers like Vioxx might be dangerous to the heart.

In 2004, editors of BMJ and Lancet called for an international register of all clinical trials. In September 2004, Pharmaceutical Research and Manufacturers of America (PhRMA) decided to create a site for companies to post the results of completed trials and launched ClinicalStudyResults.org.

Stricter Guidelines for Reporting

But what’s happening to improve the odds that drugs that get approved are at best effective and at least harmless? In 2005, the major drug companies agreed to expand the number of trials registered on ClinicalTrials.gov, an NIH site created in 2000. It is supposed to serve as a registry of both federal and private trials from across the globe.

The underreporting of clinical trial data despite promises from the industry spurred a “repurposing” of ClinicalTrials.gov in 2007.  Changes to the Food and Drug Administration Amendments Act (FDAAA) of 2007 put in place expanded registration and reporting requirements to include most clinical trials and an enforcement provision.

Responsible parties (sponsors or principal investigators of trials) who failed to comply would be fined $10,000 for each infringement, with no upper limit. They would also be named publicly in noncompliance notices posted on ClinicalTrials.gov. Further, the FDA requires disclosure of the results within one year of the date that “the last subject was examined or received intervention for purposes of final collection of data for the primary outcome.”

Pfizer says it posts results of all its registered studies including its interventional and noninterventional trials. Summary data is posted on PhRMA’s ClinicalStudyResults.org. Pfizer also posts detailed results of hypothesis-testing studies (Phase II–IV) completed on or after September 27, 2008, on ClinicalTrials.gov.

Incomplete Publishing Leads to Bias

“There are a lot of public statements from drug companies saying that they support the registration of clinical trials or the dissemination of trial results, but the devil is in the details,” noted Deborah Zarin, M.D., director of ClinicalTrials.gov.

“Our current evidence base contains incomplete and questionable evidence,” Dr. Wieseler said. “We know that not all studies that investigate drugs are published. The published literature tends to overestimate benefits and underestimate harms because positive studies are published more often and earlier than negative studies. Therefore, the picture of a drug in the published literature may be biased.”

A paper published this August in Annals of Internal Medicine by researchers from the U.S. and Canada looked at 546 drug trials registered on ClinicalTrials.gov.  Sixty three percent, or 346 of them, were funded by the drug industry. The remaining 200 were paid for by government or nonprofit organizations. 

Study authors found that more than 85% of industry-funded trials in their sample posted favorable outcomes and were four times more likely to report findings that favored their drug. The investigators reached the conclusion that studies funded by industry were less likely to be published within two years of study completion and were more likely to report positive outcomes than trials funded by other sources.

“We did this study in order to determine whether there is an inherent bias because pharmaceutical companies fund trials on products in which they have a financial interest,” said study co-author Kenneth Mandl, M.D., of Children's Hospital, Boston. “The most reassuring result would have been that the rate of favorable outcomes would be the same regardless of funding sources.”

Dr. Zarin pointed out that most of the trials used in this study were registered prior to the enactment of FDAAA and therefore were not covered by newer, more stringent registration requirements.
“It is clear, however, that some level of selective publication is occurring so that the public as well as medical experts only have access to a biased sample of studies for any given class of drugs,” Dr. Zarin conceded. “It is therefore urgent that all sponsors of trials, whether government, industry, or others, make strenuous efforts to implement the requirements under FDAAA.”