No publication without confirmation

["Dave Farber" <dave () farber net>]

---------- Forwarded message ---------
From: Dewayne Hendricks <dewayne () warpspeed com>
Date: Mon, Feb 27, 2017 at 2:13 AM
Subject: [Dewayne-Net] No publication without confirmation
To: Multiple recipients of Dewayne-Net <dewayne-net () warpspeed com>


No publication without confirmation
Proposing a new kind of paper that combines the flexibility of basic
research with the rigour of clinical trials.
By Jeffrey S. Mogil & Malcolm R. Macleod
Feb 22 2017
<http://www.nature.com/news/no-publication-without-confirmation-1.21509>

Concern over the reliability of published biomedical results grows
unabated. Frustration with this 'reproducibility crisis' is felt by
everyone pursuing new disease treatments: from clinicians and would-be drug
developers who want solid foundations for the preclinical research they
build on, to basic scientists who are forced to devote more time and
resources to newly imposed requirements for rigour, reporting and
statistics. Tightening rigour across all experiments will decrease the
number of false positive findings, but comes with the risk of reducing
experimental efficiency and creativity.

Bolder ideas are needed. What we propose here is a compromise between the
need to trust conclusions in published papers and the freedom for basic
scientists to explore and innovate. Our proposal is a new type of paper for
animal studies of disease therapies or preventions: one that incorporates
an independent, statistically rigorous confirmation of a researcher's
central hypothesis. We call this large confirmatory study a preclinical
trial. These would be more formal and rigorous than the typical preclinical
testing conducted in academic labs, and would adopt many practices of a
clinical trial.

We believe that this requirement would push researchers to be more
sceptical of their own work. Instead of striving to convince reviewers and
editors to publish a paper in prestigious outlets, they would be
questioning whether their hypotheses could stand up in a large,
confirmatory animal study. Such a trial would allow much more flexibility
in earlier hypothesis-generating experiments, which would be published in
the same paper as the confirmatory study. If the idea catches on, there
will be fewer high-profile papers hailing new therapeutic strategies, but
much more confidence in their conclusions.

The confirmatory study would have three features. First, it would adhere to
the highest levels of rigour in design (such as blinding and
randomization), analysis and reporting. Second, it would be held to a
higher threshold of statistical significance, such as using P values of P <
0.01 instead of the currently standard P < 0.05. Third, it would be
performed by an independent laboratory or consortium. This exceeds the
requirements currently proposed by various checklists and funders, but
would apply only to the final, crucial confirmatory experiment.

Unlike clinical studies, most preclinical research papers describe a long
chain of experiments, all incrementally building support for the same
hypothesis. Such papers often include more than a dozen separate in vitro
and animal experiments, with each one required to reach statistical
significance. We argue that, as long as there is a final, impeccable study
that confirms the hypothesis, the earlier experiments in this chain do not
need to be held to the same rigid statistical standard.

This would represent a big shift in how scientists produce papers, but we
think that the integrity of biomedical research could benefit from such
radical thinking.

[snip]

Dewayne-Net RSS Feed: <http://dewaynenet.wordpress.com/feed/>



-------------------------------------------
Archives: https://www.listbox.com/member/archive/247/=now
RSS Feed: https://www.listbox.com/member/archive/rss/247/18849915-ae8fa580
Modify Your Subscription: https://www.listbox.com/member/?member_id=18849915&id_secret=18849915-aa268125
Unsubscribe Now: 
https://www.listbox.com/unsubscribe/?member_id=18849915&id_secret=18849915-32545cb4&post_id=20170227024122:1F98AF70-FCC0-11E6-8656-FD88084C1F79
Powered by Listbox: http://www.listbox.com