We wish to thank Dr. Cata and Dr. Sessler for their carefully considered review of factors contributing to the failure of promising preclinical study findings to be confirmed in subsequent large clinical trials of the effects of anesthetic technique on cancer surgery outcomes.  Their forensic analysis of the reasons focused on experimental methodology in animal research, and the mechanistic drivers and reductionist approach that characterize preclinical science. Their explanation also included “often unjustified” small sample sizes and “unsophisticated” statistical analysis. (A common example might be failure to correct for multiplicity in experimental comparisons when determining statistical significance, for instance.) They pointed to the “huge effort and the expense” of the clinical trials prompted by this preclinical work, but ultimately with negative findings.

However, the authors did not mention the likely contribution of positive publication bias to this failure in research translation. The damaging effect of publication bias on evidence quality in medicine has been lamented for many years by experts who estimate half of research activity goes unpublished.  In the field of anesthesiology, we and other authors have confirmed this.  We found that, among more than 1,000 abstracts of clinical trials presented at the annual American Society of Anesthesiologists (ASA; Schaumburg, Illinois) meeting from 2001 to 2004, those with positive findings were twice as likely (odds ratio 2.01) to proceed to eventual peer-reviewed journal publication as those with negative findings.  The hallmarks of clinical research that promote publication bias are single center, small trial design, and lack of transparency in prospective hypothesis formation with potential for selective outcome reporting.  These features can all be shared by animal laboratory research, and there is certainly no reason to believe that it is any less prone to publication bias than clinical trials.

With a view to future solutions, we repeated our analysis by using ASA abstracts from 2010 to 2016, following the implementation by the International Committee of Medical Journal Editors (ICMJE) of mandatory prospective clinical trial registration for publication after 2005. We found that the magnitude of imbalance in publication outcome had significantly reduced (odds ratio 1.3), suggesting that this policy successfully limits the influence of publication bias in clinical anesthesiology.  However, mandatory prospective trial registration does not extend to preclinical trials and animal studies, nor to the publication policies of journals in other fields of the biologic sciences. Leaders in preclinical science have recently called for integration of trial registration into the animal ethics review process.  Animal research registries have been established, but with only sporadic use. 

We fear therefore that, in combination with the factors which Dr. Cata and Dr. Sessler have described, the persistence of publication bias offers little prospect that the problem of false promises by preclinical research will be solved any time soon. We can thus expect much wasted cost and effort in subsequent clinical trials to continue into the future.