Author: Michael Vlessides
Although recent years have seen marked improvement in the quality and consistency of pain research and reporting, there is still more work to do, according to a Danish research team. Their analysis concluded that although the risk for bias in randomized controlled trials has decreased over the past decade, sample sizes did not change significantly during that period, leading them to urge caution in interpreting study results, especially from older investigations.
“Each of the seven Cochrane bias domains, along with publication bias itself, can exaggerate intervention effects and create false significance in randomized controlled trials,” said Anders P.H. Karlsen, MD, PhD, a fellow in anesthesiology at Zealand University Hospital in KØge, Denmark. “In turn, this can lead to suboptimal treatment of our patients. And in the field of pain management, which often has very soft outcomes, this can have a large impact.
“We believe the way to overcome these issues would be to make a standardized guideline in postoperative pain management after total hip and knee arthroplasty,” he added.
To help determine what has evolved in the field, Dr. Karlsen and his colleagues turned to previous data from two systematic reviews regarding pain management after total hip or knee arthroplasty (Pain 2015;156:8-30; PLoS One 2017;12:e0173107). The primary outcomes of the analysis were risk for bias and trial sample size developments over time.
The researchers calculated cumulative bias scores (range, 0-14) based on Cochrane’s seven bias domains (0=low, 1=unclear, 2=high). Dates were compared for trials published between 1990-1999 and 2010-2016. “We also used a statistical process control that is widely used in quality control studies to assess the stability of a process over time,” Dr. Karlsen said.
As he reported at the 2018 Joint World Congress on Regional Anesthesia and Pain Medicine and annual meeting of the American Society of Regional Anesthesia and Pain Medicine (abstract 4930), the study included 171 trials published between 1989 and 2016. The number of articles published each year increased substantially during that period.
It was found that the summarized risk for bias decreased between the two periods, primarily due to improved randomization and allocation concealment. Visual inspection suggested continuing improvement that started around 2007. However, trial sample size did not change significantly between the two periods.
Methodology Has Become a Focus
Specifically, the analysis revealed that adequate reporting increased from 36% in 1990-1999 to 75% in 2010-2016 for random sequence generation, from 9% to 38% for allocation concealment, from 27% to 52% for blinding of participants/personnel, and from 50% to 68% for blinding of outcome assessors. The researchers also found improvements in incomplete outcome data (from 68% to 84%), selective outcome reporting (from 77% to 85%), and for other sources of bias (from 64% to 79%).
“So what is the reason for the improvement in the summarized risk of bias?” Dr. Karlsen asked. “It may be because after the millennium, focus has increased on trial methodology. We’ve seen the CONSORT [Consolidated Standards of Reporting Trials] guidelines; we’ve seen the Cochrane methodology improve. What’s more, we now have mandatory trial protocol registration at ClinicalTrials.gov.”
But with these positive changes in trial methods, the Danish team members, as well as outside researchers, found the stagnant sample sizes surprising. William F. Urmey, MD, a clinical associate professor of anesthesiology at Weill Cornell Medical College, in New York City, did not expect the trial sample sizes to remain unchanged over time. “I would think that if we’re seeing better power analysis and design for a given outcome, it would have improved.
“I also find it interesting that you started seeing changes in 2007,” noted Dr. Urmey, who was not involved in the study. “How much do you think that reflects editorial insistence from journals that researchers adhere to better protocols and better study designs?”
“I think it has a lot to do with that,” Dr. Karlsen replied. “Remember that in 2004, it became mandatory to register trials. You could still publish a study without it, if the editorial board agrees, but I think editorial boards are more inclined than ever to procure high-quality studies.”
Despite this progress, Dr. Karlsen acknowledged there is still considerable work to be done. “We’ve seen improvement, … but we’re not nearly there yet. I and my research group believe that to overcome these issues, we need to create a standardized guideline and a mandatory core outcome set to reduce the variability in the way research is reported in the field. We’re going to need global consensus on this process.”