How to read a peptide study: design, controls and the limits of a result

Almost every claim made about a research peptide traces back to a paper. Whether the claim is worth anything depends on what kind of paper it was, how it was designed and how far the design reaches — none of which is visible in an abstract. This page is about the methods section, where how much a finding establishes is actually settled.

The hierarchy, and what each level can answer

Evidence comes in layers, each answering a narrower question than it is usually quoted for.

Nearly all published work on compounds sold for research sits in the first three rows. That is not a dismissal: a rodent model with good controls is real evidence, just evidence about that model.

The controls that make a result mean anything

Vehicle control. The comparison group receives the diluent alone, by the same route, on the same handling schedule. Without it, an effect cannot be separated from the solution, the handling or the procedure. In surgical models this extends to a sham operation, because opening an animal is itself an intervention.

Positive control. A treatment already known to produce the effect, run in parallel. It shows the model could detect an effect at all. A negative result in a model with no working positive control is uninterpretable.

Randomisation, blinding and allocation

Three procedural safeguards do most of the work of separating a finding from an expectation, and their absence in animal research is common and consequential. Randomisation decides which animal goes to which group by chance rather than by the order it came out of the cage — which matters, because the easiest animal to catch is not a random animal. Allocation concealment keeps whoever assigns groups from knowing what comes next. Blinded outcome assessment keeps whoever measures the result from knowing which group it is, and matters most for exactly the endpoints peptide studies rely on: histological scoring, behavioural rating, wound measurement, anything with a judgement in it.

Surveys of the in vivo literature repeatedly find that most published animal studies do not report random allocation or blinded assessment, and that studies lacking them report larger effects (Macleod and colleagues, PMID 26460723). When a methods section is silent on both, the honest reading is that the effect is inflated by an unknown amount.

Sample size, and why small studies mislead in a specific direction

Statistical power is the probability a study would detect a real effect of a given size. Underpowered studies obviously miss real effects. The more damaging consequence is that when one does reach significance, the effect it reports is systematically too large — only an unusually large sample estimate could have cleared the threshold. Button and colleagues found median power in neuroscience far below what anyone would accept if asked directly (PMID 23571845).

This is the mechanism behind a familiar pattern: a striking first paper with a handful of animals per group, then replications with larger groups and progressively smaller effects. Ioannidis set out the general case for why a large share of findings in low-powered, high-flexibility fields should be expected to be false (PMID 16060722).

Read the effect size and the confidence interval, not the p-value. A p-value says how surprising the data would be if nothing were happening; it does not say how big the effect is, and a result can be highly significant and trivially small.

Endpoints, and the difference between a marker and an outcome

What a study measured is often narrower than what it is reported as showing. A surrogate endpoint is a measurable marker standing in for something that matters — a gene expression level, a circulating protein, a histology score. Surrogates are quick and cheap, and useful when the link to the real outcome is established. When it is not, a change in a surrogate is a change in a surrogate.

The distinction is sharpest in tissue-repair work. Collagen deposition and tensile strength are not the same endpoint: a repair can lay down abundant collagen in disorganised fibres and be mechanically weaker than one with less. Mechanical testing is the harder measurement and the more informative.

Primary and secondary endpoints matter too. If the pre-declared primary endpoint fails and a secondary one succeeds, the result is a hypothesis for the next study rather than a finding.

Pre-specification, and why it changed

Analysing data flexibly and reporting the comparison that worked produces significant results whether or not anything is there. The response has been pre-registration: recording hypothesis, sample size, endpoints and analysis plan before collecting data, so the line between a prediction and an observation survives into the paper (Nosek and colleagues, PMID 29531091). The replication projects that prompted this — in psychology (Open Science Collaboration, PMID 26315443) and preclinical biology (Begley and Ellis, PMID 22460880; Freedman and colleagues, PMID 26057340) — found a substantial fraction of published findings did not reproduce.

Reporting standards attack the same problem from the other end. ARRIVE sets out what an animal study must report for a reader to judge it, including randomisation, blinding and how sample size was arrived at (Kilkenny and colleagues, PMID 20613859); CONSORT does the equivalent for randomised human trials (Schulz and colleagues, PMID 20332509). A paper silent on what either asks for has not given you enough to assess it.

How often animal results carry over

Systematic comparisons of animal experiments against the human trials that followed them have found agreement inconsistent — concordant in some cases, contradicted in others, and in several instances human trials proceeded where the animal evidence did not support them (Perel and colleagues, PMID 17175568; Hackam and Redelmeier, PMID 17032985). Rodents differ from people in metabolism, lifespan, scale and immune function, and an induced acute injury differs from chronic human disease in kind rather than degree.

So a compound with consistent findings across several rodent models has real preclinical groundwork and no established human effect. Both are true at once, and collapsing them is the commonest error in writing about this field.

One more thing: who did the work

Where a literature comes overwhelmingly from one group, its findings share methods, reagents, models and assumptions, and consistency between them is weaker evidence than it looks. Independent replication is what converts a consistent literature into an established one. BPC-157 is this catalogue's clearest example of a literature with that shape.

For the chemistry underneath all this, see peptides explained; for the assays in the first two rows of the table, how peptide receptor signalling is studied; for the terms, the glossary. Compounds referred to here are supplied for laboratory research use only. Nothing here is medical, dosing, or therapeutic guidance.

Common questions

Does a published paper mean a finding is established? No. Publication means the work passed peer review at one journal, which checks plausibility and reporting rather than truth. Established means replicated independently, in designs capable of detecting the effect. Most compound-specific claims in this field have the first and not the second.

Is animal evidence worthless then? Not at all — it is how most of what is known about biology was first learned, and the only ethical route to a human trial. The error is treating it as a weaker version of human evidence rather than evidence about a different system. A rat tendon healing faster is a fact about rat tendons.

What is the quickest way to judge a paper I cannot read in full? Four things in the methods: animals or replicates per group, whether allocation was randomised, whether outcome assessment was blinded, and what the control group received. If the abstract claims a lot and the methods are silent on all four, treat it as preliminary.

Selected Research References

1. Button KS, Ioannidis JPA, Mokrysz C, et al. Power Failure: Why Small Sample Size Undermines the Reliability of Neuroscience. Nature Reviews Neuroscience. 2013;14(5):365-376. pubmed.ncbi.nlm.nih.gov/23571845
2. Macleod MR, Lawson McLean A, Kyriakopoulou A, et al. Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biology. 2015;13(10):e1002273. pubmed.ncbi.nlm.nih.gov/26460723
3. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biology. 2010;8(6):e1000412. pubmed.ncbi.nlm.nih.gov/20613859
4. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials. BMJ. 2010;340:c332. pubmed.ncbi.nlm.nih.gov/20332509
5. Perel P, Roberts I, Sena E, et al. Comparison of Treatment Effects Between Animal Experiments and Clinical Trials: Systematic Review. BMJ. 2007;334(7586):197. pubmed.ncbi.nlm.nih.gov/17175568
6. Hackam DG, Redelmeier DA. Translation of Research Evidence From Animals to Humans. JAMA. 2006;296(14):1731-1732. pubmed.ncbi.nlm.nih.gov/17032985
7. Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Medicine. 2005;2(8):e124. pubmed.ncbi.nlm.nih.gov/16060722
8. Begley CG, Ellis LM. Drug Development: Raise Standards for Preclinical Cancer Research. Nature. 2012;483(7391):531-533. pubmed.ncbi.nlm.nih.gov/22460880
9. Freedman LP, Cockburn IM, Simcoe TS. The Economics of Reproducibility in Preclinical Research. PLoS Biology. 2015;13(6):e1002165. pubmed.ncbi.nlm.nih.gov/26057340
10. Open Science Collaboration. Estimating the Reproducibility of Psychological Science. Science. 2015;349(6251):aac4716. pubmed.ncbi.nlm.nih.gov/26315443
11. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The Preregistration Revolution. Proceedings of the National Academy of Sciences. 2018;115(11):2600-2606. pubmed.ncbi.nlm.nih.gov/29531091

All products sold by NovaEvum are for research and laboratory use only. They are not intended for human or animal consumption, diagnosis, treatment, cure, or prevention of any disease.