A systematic review follows a predefined process to identify relevant studies, and meta-analysis then combines their results for more precise conclusions.1 The American Society of Anesthesiologists (ASA) endorses these evidence syntheses, often categorizing those with sufficient randomized controlled trials (RCTs) as Category A: Level 1 evidence—the highest level of evidence.2 Anesthesiologists rely on such reviews to guide clinical practice, including ASA guidelines for preoperative fasting and difficult airway management.2,3 Despite their importance, systematic reviews and meta-analyses can be undermined by publication bias (PB), which arises when studies with significant findings are more likely to be published than those with neutral or negative results. This bias can distort anesthesiologists’ perceptions of true efficacy, leading to overestimation of benefits and underestimation of harms.4,5 High-profile cases highlight its impact, such as underreporting cardiovascular risks linked to rofecoxib (Vioxx) and selective publication of antidepressant trials misrepresenting drug efficacy.6,7 Proactive approaches—like searching for unpublished or “gray” literature, trial registries, and mandatory protocols—aim to counteract PB and provide anesthesiologists a more accurate evidence base. Building on our prior review, we aimed to assess whether systematic reviews in leading anesthesiology journals have improved PB evaluations and mitigation strategies.8
METHODS
Article Selection
Using the h5-index of Google Scholar Metrics, Author A.K. identified 5 top-ranked journals in the anesthesiology subcategory: Anaesthesia, Anesthesia & Analgesia, Anesthesiology, British Journal of Anaesthesia, Regional Anesthesia & Pain Medicine, and Obstetric Anesthesia Digest, although currently ranked number one on the Google h5-index, was excluded because it provides article summaries and is not indexed in PubMed.
Author A.K. conducted a PubMed search (2016–2024) for systematic reviews/meta-analyses using a method adapted from a previous study, yielding 741 articles.9 After applying PubMed’s filters for “free full text,” “systematic review,” and ‘meta-analysis,’ 395 remained. Authors A.K. and J.M. screened titles/abstracts via Rayyan, a search platform for systematic reviews, and excluded 128 articles, leaving 267 for final analysis (Figure). The search string, raw data, and extraction form were uploaded to the Open Science Framework (OSF), a freely accessible data repository—https://osf.io/4gsn9/.
Data Extraction
Two independent reviewers (A.K. and J.M.) performed data extraction in a masked, duplicate manner using a standardized form. Extracted characteristics included: (a) article title; (b) authors; (c) publication year; (d) journal name; (e) number of included studies and (f) reporting guidelines used. PB-related details were also extracted, including (a) discussion of PB; (b) whether PB was formally assessed (excluding studies explicitly stating infeasibility due to few included studies); (c) PB assessment method (with GRADE not considered a formal PB assessment); (d) funnel plot inclusion; (e) PB presence, if evaluated; (f) foreign language searches (marked “yes” if any non-English search was included); (g) reference list hand-searching; (h) gray literature searching and sources and (i) clinical trial registry search. We followed the Cochrane Handbook definition of gray literature.10 We included only meta-analyses of ≥10 primary studies to ensure sufficient power to detect asymmetry and excluded the Cochrane Library and Cochrane Central Register of Controlled Trials as gray literature or registries because, although they include published and some unpublished references, they are not prospective registration platforms.
RESULTS
We analyzed 267 articles. PB was discussed in 181 (67.8%), most frequently by the British Journal of Anaesthesia (n = 102), with the highest proportion in Anesthesia & Analgesia (n = 10/12, 83.3%). A formal PB evaluation was conducted in 156 (58.4%) articles and most frequently by the British Journal of Anaesthesia (n = 89) which also had the highest proportion (n = 89/143, 62.2%). The most common method to evaluate PB was a funnel plot (n = 145, 54.3%). PB was present in 48 (17.9%) reviews, not present in 107 (40.1%), and unknown in the remaining 112 (41.9%) reviews. Overall, 213 (79.8%) studies used PRISMA, among which 136 (63.8%) evaluated PB and 156 (73.2%) discussed PB. Of the 54 studies not using PRISMA, 20 (37%) evaluated PB and 25 (46.3%) discussed it. Over half (n = 134, 50.2%) searched gray literature, most commonly ClinicalTrials.gov. Lastly, 153 (57.3%) of reviews conducted a foreign language search and 207 (77.5%) did a hand search of reference lists for relevant reviews. Among those reviews that evaluated PB (n = 156), 128 (82.1%) hand searched reference lists (Table).
| 2007–2015 | No, n(%) | Unknown, n(%) | 2016–2024 | No, n(%) | Unknown, n(%) | P value | |
|---|---|---|---|---|---|---|---|
| Yes, n(%) | Yes,n(%) | ||||||
| Publication bias discussed | 114(55.1%) | 93(44.9%) | 181(67.8%) | 86(32.2%) | .006 | ||
| British Journal of Anaesthesia | 43(57.3%) | 32(42.7%) | 102(71.3%) | 41(28.7%) | |||
| Anaesthesia | 22(51.2%) | 19(44.2%) | 56(66.7%) | 28(33.3%) | |||
| Anesthesiology | 19(52.8%) | 17(47.2%) | 13(52%) | 12(48%) | |||
| Anesthesia & Analgesia | 24(55.8%) | 19(44.2%) | 10(83.3%) | 2 (16.7%) | |||
| Regional Anesthesia & Pain Medicine | 6 (50%) | 6 (50%) | 0 (0%) | 3 (100%) | |||
| Publication bias evaluated | 89(43%) | 118(57%) | 156(58.4%) | 111(41.6%) | .0011 | ||
| British Journal of Anaesthesia | 32(42.7%) | 43(57.3%) | 89(62.2%) | 54(37.8%) | |||
| Anaesthesia | 18(43.9%) | 23(56.1%) | 49(58.3%) | 35(41.7%) | |||
| Anesthesiology | 15(41.7%) | 21(58.3%) | 12(48%) | 13(52%) | |||
| Anesthesia & Analgesia | 21(48.8%) | 22(51.2%) | 7 (58.3%) | 5 (41.7%) | |||
| Regional Anesthesia & Pain Medicine | 3 (25%) | 9 (75%) | 0 (0%) | 3 (100%) | |||
| Funnel plot presented | 38(46.3%) | 44(53.7%) | 145(54.3%) | 122(45.7%) | .256 | ||
| Publication bias present | 34(6.4%) | 45(21.7) | 128(61.8%) | 48(17.9%) | 107(40.1%) | 112(41.9%) | .00001 |
| Gray literature search | 43(20.8%) | 164(79.2%) | 134(50.2%) | 133(49.8%) | 9.76 × 10⁻¹¹ | ||
| Foreign language search | Not examined | Not examined | 153(57.3%) | 114(42.7%) | |||
| Reference lists hand search | 172(83.1%) | 35(16.9%) | 207(77.5%) | 60(22.5%) | .166 |
Our findings show progress in mitigating PB since our 2016 analysis, when 55.1% of articles discussed PB and 43% formally evaluated it, compared to 67.8% (P = .006) and 58.4% (P = .0011) in the present study, marking a noticeable increase in authors’ awareness of potential bias in pooled estimates. Funnel plot usage grew from 46.3% to 54.3% (P = .256), and gray literature searching from 20.8% to 50.2% (P = 9.76 × 10⁻¹¹), whereas hand searching of reference lists decreased from 83.1% to 77.5% (P = .166), indicating both positive developments and areas needing further improvement. Finally, PB was identified in 17.9% of current articles, compared to 6.4% (P = .00001) in the previous analysis.
DISCUSSION
Although most reviews addressed PB and formally evaluated it, many failed to conduct thorough scrutiny, potentially limiting methodological rigor. As PB can amplify benefits and mask harms, neglecting to evaluate it may compromise patient care, given that clinical decisions depend on complete evidence. Editorial oversight and enhanced reporting transparency can further mitigate PB, illustrating the need for uniform policies across journals. Increased use of trial registries, mandatory protocol disclosure, and periodic updates of systematic reviews may be beneficial. Some reviews included gray literature and non-English sources, but these strategies were inconsistently used. Collaborative efforts among authors, editors, and peer reviewers can encourage more robust methods and detailed reporting, while open science initiatives can increase evidence integrity. Our study’s focus on prominent journals and the use of free full text may limit generalizability. In conclusion, recognition of PB in leading anesthesiology journals is growing, yet further improvements are needed. Greater adherence to PRISMA, formal PB evaluations, and inclusion of unregistered studies can strengthen evidence syntheses, maintain research integrity, and ensure best practices in anesthesiology.
REFERENCES