Research Article

Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration Database

  • Erick H. Turner mail,

    Affiliations: Department of Psychiatry, Oregon Health & Science University, Portland, Oregon, United States of America, Department of Pharmacology, Oregon Health & Science University, Portland, Oregon, United States of America, Center for Ethics in Health Care, Oregon Health & Science University, Portland, Oregon, United States of America, Behavioral Health and Neurosciences Division, Portland Veterans Affairs Medical Center, Portland, Oregon, United States of America

  • Daniel Knoepflmacher,

    Affiliation: School of Medicine, Oregon Health & Science University, Portland, Oregon, United States of America

  • Lee Shapley

    Affiliation: School of Medicine, Oregon Health & Science University, Portland, Oregon, United States of America

  • Published: March 20, 2012
  • DOI: 10.1371/journal.pmed.1001189

Reader Comments (3)

Post a new comment on this article

The mere obligation to publish all studies is not enough

Posted by StefanLeucht on 04 May 2012 at 09:21 GMT

Publication bias is a major threat for evidence-based medicine. Erick Turner and colleagues elegantly show that this problem also exists in schizophrenia as four (17%) antipsychotic drug trials were not published by the pharmaceutical industry, although they highlight that the effect was less pronounced than in their previous report on antidepressants (1). Other examples in trials on maintenance treatment with antipsychotics exist, such as a large unpublished trial comparing quetiapine with haloperidol (2). Careful meta-analysis can help because results of all four studies could be found in the FDA submission documents and were included in a review on recently introduced antipsychotic drugs (ziprasidone study 104 was excluded due to too low doses (3) ).We feel that the Turner paper on antidepressants has improved the situation and we have the impression that pharmaceutical companies publish their RCTs more consistently (e.g. EliLilly – albeit briefly - reported a study showing no difference of their new glutamatergic compound (4)) and many now have their own freely accessible trial databases were at least some key results are presented. This being said we want to emphasize that the mere obligation to publish all randomized controlled trials does not sufficiently solve the problem of publication bias: A) the FDA website is difficult to search and certain requirements of the Consort statement such as the presentation of standard deviations which are crucial for meta-analysis are not respected. B) There are more subtle forms of “publication bias” such as a recent large trial showing a superiority of adding memantine to a cholinesterase inhibitor in the widely distributed journal JAMA (5), while a negative trial from was published in a specialist journal (6). Language bias constitutes a special case of this where negative studies tend to be published in non-English languages (7). That prestigious medical journals tend to be more interested in positive findings also plays a role (8). C) There is the problem of selective reporting, i.e. that outcomes with negative findings are not presented, a phenomenon that we frequently encountered in the schizophrenia literature (9). Full results should be published either in the paper or on a web supplement. Ideally, all individual patient data of the most important findings should be made publicly available. As an intermediate step internet-based, freely accessible repositories should be created in where results on all outcomes are presented.

Stefan Leucht, MD, Stephan Heres, MD and John M Davis, MD

Stefan Leucht and Stephan Heres: Department of Psychiatry and Psychotherapy, Technische Universität München, Klinikum rechts der Isar, Ismaningerstr. 22, 81675 München, Germany

John M Davis, MD: Department of Psychiatry, University of Illinois at Chicago, 1601 W. Taylor St. 324W, Chicago, Illinois, USA.

1. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.
2. Mosholder AD. Review and evaluation of clinical data (1997) Application information. NDA 20-639. FDA website
3. Leucht S, Arbter D, Engel RR, Kissling W, Davis JM (2009) How effective are second-generation antipsychotic drugs? A meta-analysis of placebo-controlled trials. Mol Psychiatry 14:429-47
4. Kinon BJ, Zhang L, Millen BA, Osuntokun OO, Williams JE, Kollack-Walker S, Jackson K, Kryzhanovskaya L, Jarkova N; and the HBBI Study Group (2011) A multicenter, inpatient, phase 2, double-blind, placebo-controlled dose-ranging study of LY2140023 monohydrate in patients with DSM-IV schizophrenia. J Clin Psychopharmacol 31:349-55
5. Tariot PN, Farlow MR, Grossberg GT, Graham SM, McDonald S, Gergel I; Memantine Study Group (2004) Memantine treatment in patients with moderate to severe Alzheimer disease already receiving donepezil: a randomized controlled trial. JAMA 291:317-24.
6. Porsteinsson AP, Grossberg GT, Mintzer J, Olin JT; Memantine MEM-MD-12 Study Group (2008) Memantine treatment in patients with mild to moderate Alzheimer's disease already receiving a cholinesterase inhibitor: a randomized, double-blind, placebo-controlled trial. Curr Alzheimer Res 5:83-9.
7. Egger M, Zellweger-Zähner T, Schneider M, Junker C, Lengeler C, Antes G (1997) Language bias in randomised controlled trials published in English and German. Lancet 350:326-9.
8. Ahmed I, Soares KV, Seifas R, Adams CE (1998) Randomized controlled trials in Archives of General Psychiatry (1959-1995): a prevalence study. Arch Gen Psychiatry 55:754-5.
9. Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S (2005) Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry 163:185-94.

Competing interests declared: Conflict of interest:
Stefan Leucht has received honoraria as a consultant or as a member of advisory boards from Alkermes, Bristol-Myers Squibb, Eli Lilly, Janssen, Johnson and Johnson, Medavante, Roche; he has received lecture honoraria from AstraZeneca, Bristol-Myers Squibb, Eli Lilly, EssexPharma, Janssen, Johnson and Johnson, Lundbeck Institute, Pfizer, and Sanofi-Aventis; and is primary investigator of a trial for which Eli Lilly has provided medication.
Stephan Heres has received honoraria from Janssen-Cilag, Sanofi-Aventis, and Johnson and Johnson; and has accepted travel or hospitality payment from Janssen-Cilag, Sanofi-Aventis, Johnson and Johnson, Pfizer, Bristol-Myers Squibb, AstraZeneca, Lundbeck, Novartis, and Eli Lilly.
John M Davis has no conflict of interest to declare.