Negative Trial Results Given to FDA May Go Unpublished or Sanitized

SAN FRANCISCO, Nov. 25 -- Unfavorable industry-sponsored drug studies may wind up buried in the bowels of the FDA after the agency's scrutiny, never coming to light in the open medical literature, said researchers here.

But trials with results that cast a drug's safety and efficacy favorably are more likely to emerge in the literature, reported Lisa Bero, Ph.D., of the University of California San Francisco, and colleagues online in Public Library of Science Medicine.

"Not all data submitted to the FDA in support of a new drug approval were published, and there were discrepancies between original trial data submitted to the FDA and data found in published trials," wrote Dr. Bero and colleagues.

Their study evaluated 164 efficacy trials submitted to the FDA for 33 new molecular entities approved in 2001 and 2002, results from 128 were eventually published.

The researchers found that the published data and conclusions frequently did not match what had been given to the FDA:

	Results for 41 primary endpoints in the FDA filings were omitted from the published papers.
	Among 43 outcomes in the filings that did not favor the drug, 20 were never published.
	In four out of five instances in which the statistical significance of findings was given differently in the published paper versus the FDA filing, the published version was more favorable.
	In 9% of published papers, conclusions were more favorable than in the corresponding FDA filings.
	Nearly one-quarter of the trials submitted to the FDA never appeared in a scientific journal at all; the results were largely negative.

In most cases, Dr. Bero and colleagues said, the omissions and differences made the journal-reported findings more favorable to the study drug.

"Thus, the information that is readily available in the scientific literature to health care professionals is incomplete and potentially biased," they wrote.

In an accompanying commentary, An-Wen Chan, M.D., Ph.D., of the Mayo Clinic, agreed that published data from FDA-filing trials is skewed toward the most favorable results.

"Biased reporting of results from [registration] trials is particularly concerning because these journal articles are the only peer-reviewed source of information on recently approved drugs for healthcare providers, who will have had limited clinical experience with these new treatments," Dr. Chan wrote.

He noted also that the findings have cost implications, insofar as bias in published results may result in excessive use of new drugs, which tend to be among the costliest products available.

Dr. Bero and colleagues reviewed the FDA's publicly available Medical and Statistical Officer Reviews for each new drug application approved from 2001 to 2002, which contain results of all trials submitted by drug sponsors.

They identified 41 new molecular entities approved during this period. Two were later withdrawn and, for six, the FDA's trial data failed to meet the researchers' prespecified criteria for evaluability.

That left 33 drugs of a wide range of types and disease indications for inclusion in the study.

The investigators also searched the scientific literature for publications through June 2007 corresponding to the trials submitted to the FDA.

Dr. Bero and colleagues found that favorable primary outcome data and an active-control (versus placebo control only) design were powerful predictors that a trial would be published in a journal.

They calculated odds ratios of 4.77 (95% CI 1.33 to 17.06) for favorable outcomes and 3.37 (95% CI 1.02 to 11.22) for active control that a trial would eventually be published.

Types of results most frequently omitted from published papers included raw counts or means, 95% confidence intervals, and intent-to-treat findings.

The published papers also often had information not included in the FDA filings, including adverse events tables, findings based on imputed data, and uncertainties reported as P values.

"Publication bias can occur in several ways, including not publishing data at all, selectively reporting data, or framing data," they noted. "We found evidence of both lack of publication and selective reporting of data."

In discussions with investigators on unpublished trials, Dr. Bero and colleagues found no cases where companies prohibited the researchers from publishing. But they did describe two cases in which the investigators said they wanted to publish the results but had not had full cooperation from the sponsors.

"In our opinion, investigators have an ethical obligation to submit the results of their research for publication," wrote Dr. Bero and colleagues.

They said their study was limited by an inability to determine conclusively why analyses and conclusions differed between FDA filings and published papers.

The researchers also said the FDA filings were not always complete and reported data in awkward formats. As a result, primary outcomes identified in the FDA reviews could have differed from the original trial protocols.

In his commentary, Dr. Chan called for "full transparency" in trial reporting for new drugs, including complete access to study protocols and regulatory agency submissions worldwide.

"Since the interests of patients are of utmost importance, it is difficult to justify why healthcare providers and policy makers should have access to only a biased subset of information that is substantially different from that which regulatory agencies have at their disposal," he wrote.

The study was funded by the National Institutes of Health. Dr. Bero and colleagues reported no potential conflicts of interest. Dr. Chan reported a relationship with the WHO and currently chairs the SPIRIT initiative, an academic effort to standardize clinical trial designs.