Skip to content
All resources

Reading the evidence

Publication Bias Explained: The Studies You Never Get to See

AR

By

· 10 min read

A summary of published research, with citations. It states no amount, schedule or route, and makes no claim about what this compound does in a person. Supplied for laboratory research use only.

On this page · 8 sections
  1. What is publication bias?
  2. How much can it distort a literature?
  3. What is a funnel plot?
  4. How did trial registration change things?
  5. Which registered trials of compounds on this site have no published results?
  6. What does this mean for mostly preclinical literatures?
  7. How can you check for it yourself?
  8. Frequently asked questions

Publication bias is the tendency for studies with positive or striking results to be published more often, and sooner, than studies with null or negative results. The published literature then overstates how well a treatment works, because the failures sit in file drawers. It is detected indirectly, with funnel plots and by comparing trial registries against journals. For any compound, the question is not only what was published but what was registered, completed and never reported.

A half-open filing drawer full of folders and manuscripts
Illustration: the file drawer, where unpublished results go.

What is publication bias?

Researchers, sponsors and journals all have reasons to prefer a positive result. A null result is harder to write up, less likely to be accepted by a prominent journal and less useful to whoever paid for the work. None of that requires bad faith. The result is the "file-drawer problem": the studies that reach print are a filtered sample of the studies that were done.

How strong is the filter? Hopewell and colleagues' Cochrane methodology review followed cohorts of registered trials to publication. Trials with positive findings were more likely to be published than those with negative or null findings, with an odds ratio of 3.90. Assuming 41% of negative trials get published, about 73% of positive ones would be. Positive trials also tended to appear after four to five years, negative ones after six to eight.

Publication bias has a close relative inside trials: outcome reporting bias, where a trial is published but only some of its outcomes are. How to read a ClinicalTrials.gov record covers comparing registered and reported outcomes.

How much can it distort a literature?

The best-documented case is antidepressants. Turner and colleagues obtained the US regulator's reviews of 74 registered trials of 12 antidepressants, covering 12,564 patients, and matched them to journal articles. Of those trials, 31% had not been published. Of trials the regulator viewed as positive, all but one were published. Trials it judged negative or questionable were, with three exceptions, either unpublished (22) or published in a way that conveyed a positive outcome (11).

JournalsRegulator's full data set
Share of trials that appeared positive94%51%
Effect sizeInflated by 32% overall, 11% to 69% by drugReference

Nobody in that story fabricated a result. Each published paper could be accurate. The distortion came from which papers existed.

What is a funnel plot?

A funnel plot is a scatter of studies in a meta-analysis, with each study's effect estimate on one axis and its size or precision on the other. Large studies estimate the effect precisely and cluster near the true value. Small studies scatter widely on both sides. With no bias, the result looks like an inverted funnel, symmetrical about the pooled estimate.

If small studies with unfavourable results are missing, one lower corner of the funnel is empty. Egger and colleagues proposed a regression test for that asymmetry. They compared eight meta-analyses with later single large trials on the same question; four disagreed, and in every disagreement the meta-analysis showed the larger effect. Asymmetry was present in three of the four discordant pairs and in none of the concordant ones. Applied more widely, their test suggested bias in 38% of meta-analyses from leading general journals and 13% of Cochrane reviews.

Two cautions. Their own conclusion was that the test has limited power when a meta-analysis rests on a few small trials, which describes most peptide literatures. And asymmetry shows that small studies behave differently from large ones. Missing studies are one explanation; poorer methods in small studies are another. How pooled estimates are judged more broadly is in systematic review vs meta-analysis.

How did trial registration change things?

Registration attacks the problem at its source. If a trial is recorded publicly before it starts, its existence cannot be hidden later, and its planned outcomes cannot be quietly swapped. Medical journals that follow the International Committee of Medical Journal Editors require prospective registration as a condition of publishing a trial. In the United States, the Food and Drug Administration Amendments Act of 2007 (FDAAA) went further, requiring sponsors of applicable trials to post summary results on ClinicalTrials.gov within one year of completion. Those results go into the ClinicalTrials.gov results database, in a fixed format that includes every prespecified outcome and the adverse events, whether or not a paper follows.

The law has not delivered full compliance. DeVito, Bacon and Goldacre examined 4,209 trials due to report under the rule. Only 40.9% reported within the one-year deadline, and 63.8% had reported at any time. The median delay from primary completion to submission was 424 days. Industry sponsors were more likely to comply than non-industry, non-government sponsors.

Registration does appear to change what gets reported. Kaplan and Irvin looked at large cardiovascular trials funded by the US National Heart, Lung, and Blood Institute. Before 2000, 17 of 30 reported a significant benefit on the primary outcome. After 2000 only 2 of 25 did, and prior registration was strongly associated with the shift toward null results. When the outcome is declared in advance, trials report it, and fewer of them win.

Note what the US requirement leaves out. It covers "applicable" trials, which excludes phase 1 studies and many trials run entirely outside the United States. Preclinical studies have no registration requirement at all.

Which registered trials of compounds on this site have no published results?

A search of the registry for compounds discussed on this site, cross-checked against PubMed on 25 October 2026, turns up completed trials with nothing in print. Absence from PubMed is not proof of non-publication; a result may sit in a conference abstract or an unindexed journal. But none of these records links to a report of the trial, and none has posted results.

TrialCompound and designEnrolledCompletedResults postedPaper found
NCT00672074Ipamorelin, phase 2, postoperative ileus1172009NoYes, Beck and colleagues, 2014
NCT01280344Ipamorelin, phase 2 dose-finding, recovery of bowel function after resection3202014NoNone found
NCT02914665Elamipretide (SS-31), phase 2, hospitalised heart failure3082017NoNone found
NCT05162768Elamipretide, phase 3, mitochondrial disease from nuclear DNA mutations1022024NoNone found
NCT02637284BPC-157 (as PCO-02), phase 1 safety and pharmacokinetics42 (estimated)Status unknown; last updated 2015NoNone found
NCT03998514CB4211, a MOTS-c analogue, phase 1a/1b882021NoNone found

Ipamorelin shows the pattern in miniature. The smaller trial was published: a proof-of-concept study of 117 patients that found no significant difference on its key endpoint. The larger follow-on trial, nearly three times the size, has neither results nor a paper. Anyone reviewing ipamorelin's clinical record from journals alone sees the smaller study only. Ipamorelin is approved nowhere; its literature is covered in ipamorelin half-life and selectivity.

Elamipretide has an unusually complete registry footprint: many of its trials have posted results, including MMPOWER-3, which missed its primary endpoints. The two above are exceptions in a sponsor's otherwise visible record. Elamipretide holds a US accelerated approval for Barth syndrome only; see what SS-31 is.

BPC-157's only registered human trial in this table is a small phase 1 study in healthy volunteers. Its record was last updated in December 2015, two months after its listed start date, and still shows an estimated enrolment, an unknown status and no outcome. BPC-157 is approved nowhere; see BPC-157 musculoskeletal research.

CB4211's record has one linked PubMed reference, marked as derived, meaning matched automatically. It is a mouse study of MOTS-c and neuropathic pain, not a report of the trial. A linked citation is not the same thing as published results. CB4211 is a modified analogue, not MOTS-c itself, and neither is approved anywhere; see MOTS-c research.

What does this mean for mostly preclinical literatures?

For compounds whose evidence is almost all animal and cell work, there is no registry to check. Animal studies are rarely registered, and a rodent experiment that found nothing has little chance of being written up. A literature of uniformly positive rat studies is exactly what publication bias predicts, whether or not the effect is real. When most of those studies also come from one laboratory, the filter and the source coincide; see conflicts of interest in research.

Publication bias also interacts with chance. If many small studies test an effect that does not exist, roughly one in twenty will cross p < 0.05. If only those are published, the literature shows a consistent effect built from noise; see p value vs confidence interval.

How can you check for it yourself?

  • Search the registry by compound name and by any development code, not just by condition.
  • Compare completed trials with published ones. A completed trial years old with no paper and no posted results is information.
  • Open the results tab where it exists; posted results include outcomes a paper may omit.
  • In a meta-analysis, look for a funnel plot, a search of trial registries and unpublished data, and a comment on small-study effects.
  • For preclinical claims, ask whether any null or negative study exists at all. If none does, that absence is itself a warning.

Phase 1, 2 and 3 trials explained covers what each phase is meant to answer. Nothing here is medical, dosing, or therapeutic guidance.

Frequently asked questions

Is publication bias the same as fraud? No. Fraud is fabricating or falsifying data. Publication bias arises from many individually reasonable decisions: an author who does not write up a null study, a journal that prefers novelty, a sponsor that moves on. Every published paper can be honest while the literature as a whole misleads.

Does trial registration eliminate publication bias? It reduces it and makes it visible, but does not eliminate it. A registered trial can still go unreported, as the examples above show, and US results-reporting compliance has been well short of complete. Registration's main value is that a missing result can be noticed and counted.

Why are null results worth publishing? Because they stop others repeating failed work, they correct the pooled estimate in meta-analyses, and they are owed to the participants who took part. A null result from a well-powered trial is often more informative than a positive result from a small one, because it narrows the range of plausible effects.

Can a funnel plot prove that studies are missing? No. Asymmetry shows that small studies report different effects from large ones. Missing small negative studies are one cause; poorer methods in small studies, or genuine differences between the populations studied, are others. With fewer than a handful of studies, a funnel plot cannot show much either way.

Selected Research References

  1. Hopewell S, Loudon K, Clarke MJ, et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews. 2009;(1):MR000006. pubmed.ncbi.nlm.nih.gov/19160345
  2. Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine. 2008;358(3):252-260. pubmed.ncbi.nlm.nih.gov/18199864
  3. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. pubmed.ncbi.nlm.nih.gov/9310563
  4. DeVito NJ, Bacon S, Goldacre B. Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. The Lancet. 2020;395(10221):361-369. pubmed.ncbi.nlm.nih.gov/31958402
  5. Kaplan RM, Irvin VL. Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLoS One. 2015;10(8):e0132382. pubmed.ncbi.nlm.nih.gov/26244868
  6. Beck DE, Sweeney WB, McCarter MD. Prospective, randomized, controlled, proof-of-concept study of the Ghrelin mimetic ipamorelin for the management of postoperative ileus in bowel resection patients. International Journal of Colorectal Disease. 2014;29(12):1527-1534. pubmed.ncbi.nlm.nih.gov/25331030
  7. Karaa A, Bertini E, Carelli V, et al. Efficacy and Safety of Elamipretide in Individuals With Primary Mitochondrial Myopathy: The MMPOWER-3 Randomized Clinical Trial. Neurology. 2023;101(3):e238-e252. pubmed.ncbi.nlm.nih.gov/37268435

All products sold by NovaEvum are for research and laboratory use only. They are not intended for human or animal consumption, diagnosis, treatment, cure, or prevention of any disease.