From The Medscape Journal of Medicine > Psychiatry

Publication Bias and the Pharmaceutical Industry: The Case of Lamotrigine in Bipolar Disorder

S. Nassir Ghaemi, MD, MPH; Arshia A. Shirzadi, DO; Megan Filkowski, BA

Posted: 09/10/2008; Medscape J Med. 2008;10(9):211 © 2008 Medscape

Abstract

Publication bias, especially the lack of publication of negative treatment studies, is known to be a major problem in the medical literature. In particular, it appears that the pharmaceutical industry is not routinely making data from negative studies available through the published scientific literature. In this paper, we review the case of studies with lamotrigine in bipolar disorder, describing evidence of lack of efficacy in multiple mood states outside of the primary area of efficacy (prophylaxis of mood episodes). In particular, the drug has very limited, if any, efficacy in acute bipolar depression and rapid-cycling bipolar disorder, areas in which practicing clinicians, as well as some academic leaders, have supported its use. The negative unpublished data now made available on lamotrigine provide an important context for clinical practice and research, and also raise important scientific and public policy concerns about having access to studies showing inefficacy with psychotropic medications.

 

---

Reader Comments on: Publication Bias and the Pharmaceutical Industry: The Case of Lamotrigine in Bipolar Disorder
See reader comments on this article and provide your own.

Readers are encouraged to respond to the author at nghaemi@tuftsmedicalcenter.org or to Peter Yellowlees, MD, Deputy Editor of The Medscape Journal of Medicine, for the editor's eyes only or for possible publication as an actual Letter in the Medscape Journal via email: peter.yellowlees@ucdmc.ucdavis.edu

Introduction

Publication bias is known to be a major problem in the medical literature.^[1-5] In general, studies with positive outcomes are more likely to be published than studies with negative outcomes.^[3] This finding also extends to the psychiatric literature.^[6] Negative outcomes are here defined as randomized clinical trials (RCTs) which fail to find statistically significant benefit with an active treatment as opposed to a control group in an adequate sample size.^[7] The nonpublication of negative outcomes in RCTs has been especially reported to occur in pharmaceutical industry-funded studies, as opposed to studies funded by governmental sources.^[6,8-12] Recent controversies in varied medical disciplines have highlighted the importance of having access to such unpublished data; previously underappreciated unpublished side effect data have led to removal of certain agents, such as the anti-inflammatory agent rofecoxib, from the US market.^[13,14] Marketing of other agents, such as gabapentin, for unproven treatment indications such as bipolar disorder despite initially unpublished studies showing absence of efficacy in that condition also has led to controversies and even malpractice lawsuits.^[15,16] A recent review of antidepressant RCTs in unipolar depression in the Food and Drug Administration (FDA) database found that almost all negative studies were unpublished, leading to a false impression in the published literature that 93% of antidepressant RCTs had positive results; when unpublished studies were included, 51% of all RCTs were positive and 49% were negative.^[17] Thus, unpublished studies raise questions of concerns regarding both underreported risks and underreported limitations in efficacy.

In response to some of the above controversies, in 2002, the US lobbying group for the pharmaceutical industry (Pharmaceutical Research and Manufacturers Association; PhRMA) published its "Principles on the Conduct of Clinical Trials and Communication of Clinical Trial Results" (http://www.phrma.org/publications/policy_papers/phrma_clinical_trial_registry_proposal/) and committed to voluntarily registering all clinical trials at initiation, as well as providing access to the results of all study outcomes, positive or negative, via company-based (specific Web sites or www.clinicalstudyresults.org) or government-based Web sites (www.clinicaltrials.gov).

In this paper, we access those Web sites of companies with agents approved for the treatment of bipolar disorder, to examine the availability of study results with negative data, and then we focus on one of those agents with availability of negative data, lamotrigine, to examine the impact of those results on clinical interpretation of the utility of that agent in bipolar disorder.

Methods

We searched the Web sites of the following companies with FDA-approved drugs for the treatment of bipolar disorder patients: Abbott Laboratories (Depakote brand of divalproex sodium), GlaxoSmithKline (Lamictal brand of lamotrigine), Pfizer Inc. (Geodon brand of ziprasidone), Eli Lilly and Co. (Zyprexa, Symbiaxbrands of olanzapine and olanzapine-fluoxetine combination), Janssen Pharmaceutica (Risperdal and Risperdal Consta brand of risperidone), Bristol Myers Squibb (Abilify brand of aripiprazole), and AstraZeneca (Seroquel brand of quetiapine). We supplemented searches of the primary company Web sites with searches of the Web sites for each drug. If no studies were identified, or if links were provided, we accessed the Web site established by the Pharmaceutical Research Manufacturers Association (PHRMA) for clinical trial results (www.clinicaltrialresults.org) and that of the National Institutes of Health (NIH) for current clinical trials (www.clinicaltrials.gov). Table 1 describes the studies found on their Web sites, whether they were positive or negative, and whether they were previously published. Published studies of the same drugs were assessed by a computerized MEDLINE search of the National Library of Medicine (NLM) database (1966–2006).

Negative vs Failed Trials

Our use of the term "negative" to describe the outcomes of studies is defined as follows: In the strictest definition, this refers to a study in which the experimental drug is the same as placebo, and in which a proven active control is more effective than placebo.^[18] If an active control fails to separate from placebo, as well as the experimental drug, then this study would be seen as a "failed" study, meaning that no conclusion could be drawn for or against efficacy, since an already proven treatment also failed to show benefit. Usually in that case, peculiar characteristics of the sample (such as a low severity of illness leading to high placebo response) can obscure a real treatment effect.^[18] In the absence of an active control group, which is often the case for FDA regulation-oriented pharmaceutically sponsored studies, one cannot be definitively certain that equivalence of drug to placebo represents a "negative" outcome.^[18] However, if the study is adequately powered to demonstrate a modest effect size, then one can at least conclude that the study is most suggestive of lack of moderate or greater benefit.^[7] In that sense, we will define studies as negative in this report if drug is equivalent to placebo in the absence of an active control group. Further, all definitions of difference between drug and placebo will be limited to the primary outcome measures, since secondary outcome measures are associated with inflated chance of statistically significant findings (P value below .05).^[19] This limitation does not mean that secondary outcomes are always false or unhelpful, but rather one should look for multiple measures showing robust effects, rather than isolated borderline P value-based analyses.^[20]

Where the comparison is of a drug to an active control without a placebo group, usually the purpose of the study is to demonstrate "noninferiority" or "equivalence" to a proven treatment.^[7] If the experimental drug is equivalent to or better than the proven treatment, the study is considered positive. If the experimental drug is less effective then the proven treatment, the study is considered negative.^[7]

Results

Two companies providing the largest number of studies were Eli Lilly and GlaxoSmithKline (GSK); an overall summary of our results is shown in Table 1 . We then focused on the results provided for the agent with the largest number of negative studies available, lamotrigine.

Examining the Lamotrigine Data

Of major US pharmaceutical companies, so far only GSK has provided data on unpublished negative studies with results that were unfavorable to their product lamotrigine (Lamictal) ( Table 2 ).

This was not a voluntary act but rather due to legal judgment brought by the state of New York after a lawsuit about paroxetine use in children.^[21] Of the 9 studies with lamotrigine in bipolar disorder provided by the GSK Web site, 2 were positive and published, and supported the company's success in securing an FDA-approved indication for lamotrigine for delay of relapse in the long-term treatment of bipolar disorder patients.^[22,23] Two negative studies have been published, 1 in rapid-cycling^[24] and another in acute bipolar depression,^[25] but both published versions emphasize positive secondary outcomes as opposed to the negative primary outcomes. A negative study in rapid cycling has not been published in detail (GW611), nor have 2 negative randomized studies in acute bipolar depression (GW 40910 and GW 603), or 2 negative randomized trials in acute mania (GW609 and GW 610). However, all of these negative studies are reported at the company Web site, which also refers to a publication^[26] that briefly summarizes results from the above 5 negative studies. However, that publication provides little actual data from these negative studies; the results are much more fully provided on their Web site.

Recently, using meta-analysis, academic investigators have published the results of 5 negative studies with lamotrigine in acute bipolar depression in more detail, confirming lack of benefit for primary outcomes.^[27] When pooling those 5 studies to produce a large sample of over 2000 subjects, a small effect size of depressive symptom benefit was statistically significant. However, more benefit was seen in a subgroup analysis of those with high severity of depressive symptoms.^[27]

Discussion

Given the above analysis, it still appears that negative outcomes, instead of being seen as important results in their own right, are treated by the pharmaceutical industry and many academic authors as if they represent signets, putting a seal on all further discussion, thus needing to be suppressed or explained away. Even though GSK has now made the studies available, the information has not been widely appreciated by the medical community.

The clinical relevance of the specific results with lamotrigine that are available through the GSK Web site is not insignificant. Two non-industry-funded studies found some benefit with lamotrigine in acute bipolar depression (1 published^[28] and another recent one not yet published).^[29] Yet the unpublished negative studies described here are needed to put such positive studies in context. Many clinicians and academics widely view lamotrigine as an effective treatment for, colloquially, "bipolar depression,"^[30] even though the entire literature is less consistent on this point than many clinicians and academics appear to assume. This impression has been furthered by the selective publishing of one of the RCTs^[25] with a positive secondary outcome on Montgomery Asberg Depression Rating Scale without emphasizing its negative outcome on the study's primary outcome on Hamilton Depression Rating Scale. Further, numerous review papers of lamotrigine in bipolar disorder cite the single published study as evidence of benefit in acute bipolar depression, but make no mention of the unpublished studies.^[30-32] This problem takes nothing away from the evidence of lamotrigine's benefit in prophylaxis of bipolar disorder, particularly in prevention of depressive episodes, although it should be noted that the benefits seen in those prophylaxis studies are limited to patients who initially tolerated and appeared to respond to lamotrigine for acute mood episodes. But preventive benefit does not necessarily translate to acute efficacy, or vice versa.^[33] Clinicians must learn to distinguish treatment of acute bipolar depression from its prevention. The lamotrigine data, positive and negative, could have been an important means of providing empirical evidence of the importance of this distinction.

The importance of academic access to such unpublished data can hardly be underestimated. For instance, the published study of lamotrigine in rapid cycling bipolar disorder is conservatively reported: the negative result in the primary outcome is acknowledged in the abstract, and the benefit in a secondary outcome in type II bipolar disorder may be clinically plausible and important. However, despite this acceptable presentation of this study, the nonpublication of the second RCT, where there was no indication at all of any benefit in type II rapid cycling bipolar disorder, leaves a mistaken impression in the published literature that lamotrigine is likely effective in that population. One out of 1 positive RCTs provides a different context than 1/2 positive RCTs. Instead, post-hoc exploratory analyses of the unpublished data are published, which, after extensive data analysis, find some possible benefit (somewhat more euthymia in lamotrigine-treated patients vs placebo utilizing a secondary outcome measure, the Life Chart Methodology).^[34] While these exploratory analyses might be relevant, and even correct, the nonpublication of the straightforward findings of the original RCT leads to a distorted context whereby the psychiatric literature overestimates the benefit of the drug.

Further, it is well known that systematic reviews, especially meta-analyses, are vulnerable to major biases if negative or failed trials showing statistical nonseparation of a drug vs placebo or an active comparator are not available for analysis.^[4,35] Further, narrative reviews of agents for bipolar disorder will provide the false impression that certain agents, such as lamotrigine, have been consistently effective whenever they were studied, since positive studies are evidently more likely to be reported at Web sites or in published articles. The practice of burying negative data in infrequently read review papers seems to be one approach to "publishing" such data, as exemplified by the single GSK-sponsored review paper which briefly summarized 5 negative studies with limited detail^[26]; yet this approach is not likely to provide adequate information for clinicians and researchers. Even when the negative studies are made available in more detail, as with a recent meta-analysis of 5 negative acute bipolar depression studies with lamotrigine,^[27] that kind of analysis should not replace earlier availability of each specific study. Further, since meta-analysis is an observational data analytic exercise, it too is not definitive, and is open to differing interpretations.^[36] Too often, it can take the form of "statistical alchemy,"^[37] whereby huge samples provide statistical significance to clinically meaningless differences. There is some danger of this kind of interpretation of the recent lamotrigine meta-analysis.^[27]

Attention to publishing negative RCTs in a straightforward manner might be less necessary if clinicians were more critical readers of the research literature. However, many clinicians appear unable to exercise methodological critique in their evaluation of papers, thus not sufficiently realizing, for instance, that the 2 main published studies of lamotrigine in acute bipolar depression needed to be interpreted with much caution (one used a crossover design,^[28] and the other emphasized a positive secondary outcome^[25]). Thus, publishing negative data, though an improvement on the status quo, will not be a panacea; clinicians will still need more training in critically interpreting research.

The clinical relevance of the lamotrigine studies is notable: Taking the negative outcomes into account, as of now, one might say that this agent is reasonably effective in maintenance treatment of bipolar disorder, particularly in prevention of depression, among patients who initially tolerate and may benefit from acute lamotrigine treatment. It is proven ineffective in acute mania, rapid cycling, and acute bipolar depression. Exploratory analyses suggest that a small effect size of benefit might occur in some patients with bipolar depression,^[27] perhaps among those more severely ill,^[27] and the hypothesis of possible benefit in type II rapid cycling bipolar disorder was raised by one study but disconfirmed by another. This context of where the drug is effective, and where it is not, is vital for scientifically valid and ethically honest clinical practice and research.

Conclusions

The pharmaceutical industry is not routinely making data from negative studies available through the published scientific literature or in available Internet sources. In the case of bipolar disorder, the negative unpublished data made available on lamotrigine provide an important context for clinical practice and research.

Medscape J Med. 2008;10(9):211 © 2008 Medscape