Meta-analyses of Randomized Clinical Trials in Postsurgical Pain: Verify before Trusting

Postsurgical pain trials often have mixed outcomes because of the complexities in measuring pain and the need for concomitant analgesics. Multimodal analgesic regimens are a mainstay of clinical practice and, when incorporated into a clinical trial, make it difficult to determine the impact of a single analgesic when not adequately controlled. Ethical guidelines mandate that randomized participants have access to rescue medication when the study intervention does not provide sufficient pain relief.  Consequently, observed pain intensity scores in postsurgical pain randomized clinical trials reflect a combination of the analgesic effects of the randomized intervention plus those of additional analgesics received.

In the context of providing adequate evidence of efficacy, the U.S. Food and Drug Administration (Silver Spring, Maryland) cautions sponsors of randomized clinical trials as follows: “Protocols should also prespecify the frequency, amount, and threshold of pain at which allowable rescue medication(s) can be administered. This is particularly important in placebo-controlled trials where increased use of rescue medication in the control group may diminish the study drug’s treatment effect, leading to a conclusion of ineffectiveness.”  This issue is relevant for active-controlled trials as well. Patients randomized to the less effective intervention will take more rescue analgesics, which will decrease the pain intensity difference between the treatments.  Thus, when estimating effectiveness, use of rescue and pain intensity scores must be considered together; pain intensity must be adjusted on the basis of the use of rescue medication to ascertain an unbiased estimate of the treatment effect of the study intervention, which is essential in determining clinical significance. 

While the Food and Drug Administration would not consider evidence that does not include adequately performed adjustment to demonstrate efficacy of an investigational drug, this issue has been broadly ignored in independent studies, and consequently meta-analyses. When meticulously collected patient-level analgesic data (e.g., type, dose, frequency) are not available to appropriately adjust pain intensity for rescue analgesics in randomized clinical trials, effect size will be attenuated. This is apparent in many postsurgical pain studies, as evidenced by a systematic review that concluded regional anesthesia techniques may be of limited value.  Thus, it may not be possible to derive unbiased estimates of analgesic effectiveness in a meta-analysis unless the same imputation procedure is applied in each trial. These questions warrant further study and underscore the importance of appropriate statistical methodologies to analyze postsurgical pain trials.

With the variety of clinical contexts, analgesic regimens, and methods available to investigators in postsurgical pain research, both clinical and methodologic heterogeneity are inevitable when conducting a meta-analysis. The ability to handle heterogeneity across studies is one of the primary advantages of a meta-analysis. However, if inclusion criteria of a meta-analysis are too broad, substantial differences in trial designs can impair the internal validity of the findings.¹¹ 

The studies included in the meta-analysis by Hussain et al. evaluated liposomal bupivacaine in a variety of abdominal facial plane blocks, using different approaches and injectate volumes (range, 30 to 200 ml) with or without supplemental bupivacaine or adjuvants and in a variety of procedures and patient populations (colorectal procedures, ventral hernia repair, hysterectomy, cesarean section, breast reconstruction surgery, Roux-en-Y gastric bypass, donor nephrectomy, or unspecified) and surgical approaches (laparoscopic, open, or robotic), as well as a range of multimodal regimens and rescue medications. As expected, substantial heterogeneity in the primary results was observed, which the authors acknowledged and attempted to address using subgroup and sensitivity analyses. However, there are limits to what statistical procedures can do to address clinical variability, which calls into question whether a meta-analysis is appropriate in the first place.

Even when applied in a narrow clinical context, there are several approaches to meta-analyses that can result in a broad range of outcomes depending on the analytic method applied. This point was well illustrated in a study by Dechartres et al., which evaluated 163 meta-analyses published in high-impact journals that employed a variety of approaches resulting in a broad range of “best estimates.” These approaches go beyond whether to include a study assumed to be conflicted by the funding source, to focus on established factors known to affect findings, such as sample size and risk of methodologic bias. Striking an appropriate balance between trial inclusivity and heterogeneity is critical for ensuring the validity and generalizability of meta-analyses in postsurgical pain.

Appraisal of bias is necessary in the conduct and reporting of a meta-analysis.  Concerns often focus on potential bias associated with industry conduct or support of randomized clinical trials. While this skepticism is warranted because of potential conflicts of interest, too often overlooked are the biases and risks from research studies that lack robustness and are improperly designed or analyzed. Researchers at McGill University (Montreal, Canada) and Brigham and Women’s Hospital and Harvard (Boston, Massachusetts) found that industry-sponsored intervention trials were considerably more likely to be informative for clinical practice versus those not sponsored by industry (50% vs. 6%).  The researchers attributed this finding to differences in the ability to complete enrollment and publish results. Postsurgical pain randomized clinical trials are challenging to conduct, requiring sufficient resources to enroll participants, conduct appropriate research assessments, and appropriately analyze data.

In the study by Hussain et al., 3 of the 12 trials included in the primary meta-analysis were terminated prematurely, with 2 trials (NCT02179892 and NCT02652156) enrolling fewer than 10 participants in total across both treatment arms, yet none of these trials were deemed to have a high risk of bias due to missing outcome data. In fact, two were deemed to have a low risk of bias, and one had only some concerns in this domain.  Overall risk of methodologic bias was deemed low in two of the three trials (NCT0363863 and NCT02652156), and there were only some concerns overall for NCT02179892; all this despite one trial being unblinded (NCT02652156), another being single-blinded (NCT0363863), and yet another having no protocol available (NCT02179892). Nevertheless, aggregate findings from these studies reported on ClinicalTrials.gov were included in the meta-analysis and given equal weight in the overall outcome. The quality of results obtained by pooling data is highly influenced by the quality of the included studies. Insufficiencies of the individual randomized clinical trials cannot be corrected at the meta-analysis stage.

In summary, meta-analyses play a primary role in evidence-based medicine, yet several pervasive challenges can undermine their interpretability and run the risk of negatively impacting patient care. There are limits to what a statistical procedure can do to address data that are inherently flawed and/or clinically heterogenous. Striking a balance between inclusivity of high-quality trials and appropriate analytic methodology is critical to ensure the validity and generalizability of meta-analyses. Authors should use caution in drawing conclusions even when essential limitations are acknowledged. Clinicians should appreciate that overstated conclusions can be misleading and run the risk of negatively impacting evidence-based decision-making and ultimately patient care.