Currently, the quality of guidelines for the perioperative management of patients with obstructive sleep apnea (OSA) is unknown, leaving anesthesiologists to make perioperative management decisions with some degree of uncertainty. This study evaluated the quality of clinical practice guidelines regarding the perioperative management of patients with OSA. This study was reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A systematic search of the MedlineALL (Ovid) database was conducted from inception to February 26, 2021, for clinical practice guidelines in the English language. Quality appraisal of guidelines was evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) framework. Descriptive statistical analysis of each of the 6 domains was expressed as a percentage using the formula: (obtained score – minimum possible score)/(maximum possible score – minimum possible score). Of 192 articles identified in the search, 41 full texts were assessed for eligibility, and 10 articles were included in this review. Intraclass correlation coefficients of the AGREE II scores across the 7 evaluators for each guideline were each >0.9, suggesting that the consistency of the scores among evaluators was high. Sixty percent of recommendations were based on evidence using validated methods to grade medical literature, while the remainder were consensus based. The median and range scores of each domain were: (1) scope and purpose, 88% (60%–95%); (2) stakeholder involvement, 52% (30%–82%); (3) rigor of development, 67% (40%–90%); (4) clarity of presentation, 74% (57%–88%); (5) applicability, 46% (20%–73%); and (6) editorial independence, 67% (19%–83%). Only 4 guidelines achieved an overall score of >70%. This critical appraisal showed that many clinical practice guidelines for perioperative management of patients with OSA used validated methods to grade medical literature, such as Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) and Oxford classification, with lower scores for stakeholder involvement due to lack of engagement of patient partners and applicability domain due to lack of focus on the complete perioperative period such as postdischarge counseling. Future efforts should be directed toward establishing higher focus on the quality of evidence, stakeholder involvement, and applicability to the wider perioperative patient experience.
Obstructive sleep apnea (OSA) is the most common form of sleep-disordered breathing seen in the perioperative setting. It is associated with higher rates of postoperative complications, increased risk of morbidity and mortality, and increased resource utilization. Despite having a high prevalence in the surgical population, up to 60% to 90% of patients with OSA undergoing surgery remain undiagnosed in their perioperative course.1–5
The high prevalence of OSA in the surgical population has incentivized the development of guidelines for risk management and mitigation. Different societies have published several clinical practice guidelines (CPGs) to guide anesthesiologists on the management of these patients with diagnosed or suspected OSA. Many guidelines use OSA-screening questionnaires, such as the STOP-Bang (Snoring, Tiredness, Observed apnea, blood Pressure, Body mass index, Age, Neck circumference and Gender) Questionnaire, the Perioperative Sleep Apnea Prediction Score (P-SAP), the Berlin Questionnaire, and the American Society of Anesthesiologists (ASA) checklist. These have been validated for preanesthetic risk stratification.6–8 Similarly, patients with a diagnosis or high suspicion for OSA should be counseled about the heightened perioperative risk and mitigation strategies that may be implemented to ensure perioperative safety. This can include modifications to preoperative interventions, such as initiation of positive airway pressure therapy and weight loss. Intraoperative techniques may prioritize regional anesthesia over general anesthesia. Finally, postoperative management may include reducing systemic opioid use, postoperative noninvasive positive pressure ventilation, and admission to a monitored setting. Despite the overwhelming clinical evidence associating the perioperative risks of patients with OSA, evidence supporting perioperative safety measures is largely lacking. Consequently, clinicians may choose to rely on guidelines published by perioperative societies and expert groups to guide management strategies.
Currently, the quality of recommendations for perioperative management of patients with OSA is unknown, often leaving anesthesiologists the difficult task of making decisions regarding preoperative preparation, intraoperative management, and postoperative course with some degree of uncertainty. This study evaluated the quality of CPGs regarding perioperative management of patients with OSA using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.
This study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.9 A systematic search of the MedlineALL (Ovid) database was conducted from inception to February 26, 2021, for CPGs. The search used controlled vocabulary and text words for terms and synonyms for the component blocks of: “sleep apnea” AND “perioperative” and “guidelines.” The search strategy was designed and performed by an informatics specialist. The full-search strategy is shown in Supplemental Digital Content 1, Table 1, https://links.lww.com/AA/E148.
Guidelines were selected according to the following inclusion criteria using a 2-reviewer system: (1) guidelines concerning the perioperative management of patients with OSA undergoing surgical procedures; (2) guidelines developed by multiple authors or medical organizations; and (3) guidelines available in full text in the English language. We excluded unpublished guidelines, consensus opinions, systematic reviews, functional algorithms, editorials, institution protocols, or surgical technique guidelines.
Data were extracted independently from all eligible articles using a predetermined data collection tool. Panelists were selected by experts from the Society of Anesthesia and Sleep Medicine (SASM), which includes the Society for Ambulatory Anesthesia (SAMBA), the Society of Critical Care Anesthesiologists (SOCCA), and patient advocacy groups, all of which work to develop evidence-based guidelines for the perioperative management of patients with OSA.10 Panelists had expertise in perioperative management of OSA and consensus guideline development, with training in anesthesiology, critical care medicine, sleep medicine, and clinical epidemiology. Using an online platform (My AGREE PLUS) allows evaluators to complete and record online AGREE II appraisals.11 The AGREE II platform allows evaluators to: (1) complete individual appraisals; (2) contribute to group appraisals; and (3) coordinate a group appraisal. The online appraisal took place between March 3, 2021, and March 18, 2021.
|Study #||Practice guideline||Year||Society||Funding/conflicts of interest||Grading system||Development method||Overall recommendations (7 reviewers)|
|1||Evaluation and management of obesity hypoventilation syndrome||2019||American Thoracic Society||Reported||GRADE||Evidence-based||6||1||0|
|2||Preoperative screening and assessment of adult patients with OSA||2016||Society of Anesthesia and Sleep Medicine||None||GRADE||Evidence-based||5||2||0|
|3||Diagnosis and treatment of sleep-disordered breathing in adults||2011||Canadian Thoracic Society||Not reported||None||Consensus-based||5||2||0|
|4||Intraoperative management of adult patients with OSA||2018||Society of Anesthesia and Sleep Medicine||Reported||GRADE||Evidence-based||5||2||0|
|5||Perioperative care of patients with OSA undergoing upper airway surgery||2019||None||Reported||GRADE||Evidence-based||5||2||0|
|6||Perioperative OSA management in bariatric surgery||2017||None||Reported||GRADE||Consensus-based||4||2||1|
|7||Preoperative selection of adult patients with OSA scheduled for ambulatory surgery||2012||Society for Ambulatory Anesthesia||Reported||SIGN||Evidence-based||3||4||0|
|8||Perioperative management of patients with OSA||2014||American Society of Anesthesiologists||Not reported||None||Evidence-based||2||4||1|
|9||Anesthetic perioperative care and pain management in weight loss surgery||2009||None||Reported||None||Consensus-based||1||4||2|
|10||Diagnosis and treatment of sleep-disordered breathing||2006||Canadian Thoracic Society||Reported||None||Consensus-based||1||2||4|
Other data variables extracted included publication year, publishing society, target population, time period (preoperative, intraoperative, and postoperative), research questions or objectives, development method of guidelines (consensus-based and evidence-based), funding or conflicts of interest (reported and not reported), and grading of evidence system (eg, Oxford classification and Grading of Recommendations, Assessment, Development, and Evaluation [GRADE] framework) used.12
Appraisal of Guidelines for Research and Evaluation II Instrument
Quality appraisal of guidelines was evaluated using the AGREE II instrument, a validated assessment tool that appraises the quality of CPGs.13 It includes 23 items across 6 domains: (1) scope and purpose, addressing the overall aim of the guideline; (2) stakeholder involvement, focusing on the incorporation of appropriate stakeholders in development of the guideline; (3) rigor of development, concerning the methodology used to gather and synthesize evidence and formulate recommendations; (4) clarity of presentation, evaluating the language, format, and structure of the guideline; (5) applicability, relating to the barriers and facilitators to implementing the guideline; and (6) editorial independence, concerning the development of the guideline without bias from competing interests. Additionally, there are 2 global rating items of overall assessment, including the rating of overall quality of the guideline and the potential recommendation for use of the guideline in clinical practice. Each item is rated using a Likert scale from 1 (strong disagreement) to 7 (strong agreement). Each domain-scaled score is scored as defined in the AGREE II instrument manual by summing the rating of individual items in each domain and standardizing them using the following formula: (obtained score – minimum possible score)/(maximum possible score – minimum possible score). Appraisal was performed independently by 7 authors to assess the reliability of the assessment. All appraisers completed training on the AGREE II tool before evaluating eligible guidelines.
The primary outcome was the overall rating of the quality of the guidelines using the AGREE II tool (domain 7). Secondary outcomes included individual ratings of each of the AGREE II domains (domains 1–6).
The median and range scores for each domain were calculated and expressed as a percentage, using the above formula. A sample size of 7 evaluators was determined based on AGREE II instructions and previous studies that have similarly used the AGREE II tool to assess CPGs to achieve sufficient power for the analysis.13–16 Consistency of scores by our 7 evaluators was estimated using a 2-way mixed-effects model for the intraclass coefficient (ICC) analysis, assuming a group of k raters was randomly selected from a population and then used to rate subjects. The following ICCs were assigned to a level of consistency: (1) <0.40, poor agreement; (2) 0.40–0.54, weak agreement; (3) 0.55–0.69, moderate agreement; (4) 0.70–0.84, good agreement; and (5) 0.85–1.0, excellent agreement. A P value of <.05 was considered statistically significant. Statistical analysis was performed using IBM SPSS Statistics version 23 (IBM Corp).17
A total of 192 articles were identified in the initial search. After title and abstract screening, 41 full texts were assessed for eligibility. After review of the full texts, 10 articles met inclusion criteria for this review (Figure).18–27 Guideline characteristics are shown in Table 1. Nine of the 10 guidelines were published by groups from North America, and one was published by a group from the Netherlands. Six of the 10 guidelines used an evidence-based method, with 5 guidelines using the GRADE framework (Table 1).
AGREE II Domain Scores
Domain scoring from the AGREE II instrument is shown in Table 2. The median and range of each domain are described below:
|Study #||Practice guideline||Scope and purpose (%)||Stakeholder involvement (%)||Rigor of development (%)||Clarity of presentation (%)||Applicability (%)||Editorial independence (%)||Overall assessment (%)|
|1||Evaluation and management of obesity hypoventilation syndrome||92||82||90||88||73||83||85|
|2||Preoperative screening and assessment of adult patients with OSA||95||60||71||87||55||67||73|
|3||Diagnosis and treatment of sleep-disordered breathing in adults||90||63||82||83||54||67||73|
|4||Intraoperative management of adult patients with OSA||94||50||71||82||47||75||70|
|5||Perioperative care of patients with OSA undergoing upper airway surgery||90||60||67||76||45||67||68|
|6||Perioperative OSA management in bariatric Surgery||78||54||65||72||48||59||63|
|7||Preoperative selection of adult patients with OSA scheduled for ambulatory surgery||86||34||67||70||40||68||61|
|8||Perioperative management of patients with OSA||76||45||47||57||29||58||52|
|9||Anesthetic perioperative care and pain management in weight loss surgery||67||35||42||60||40||63||51|
|10||Diagnosis and treatment of sleep-disordered breathing||60||30||40||69||20||19||40|
|Median (range)||88 (60–95)||52 (30–82)||67 (40–90)||74 (57–88)||46 (20–73)||67 (19–83)||65 (40–85)|
(1) Scope and purpose, 88% (60%–95%). The median score on this domain was the highest among all domains. All CPGs scored >60%, which indicates that the overall objective, clinical questions, and patient populations were specifically described (Supplemental Digital Content 2, Table 2, https://links.lww.com/AA/E149).
(2) Stakeholder involvement, 52% (30%–82%). This domain received the second lowest median score. Two of the guidelines scored >60%.22,23 The American Thoracic Society (ATS) guideline, which included a patient in the guideline panel to provide perspective on their values and preferences, received the highest score of 82%.22 This CPG also involved a methodologist and scholars to participate in the guideline discussion.
(3) Rigor of development, 67% (40%–90%). Most guidelines used systematic methods to search for evidence, although many lacked a procedure for updating the guideline or did not update guidelines in consideration of emerging evidence.
(4) Clarity of presentation, 74% (57%–88%). Most guidelines performed well in this domain, with clear presentation of recommendations (Supplemental Digital Content 3, Table 3, https://links.lww.com/AA/E150).
(5) Applicability, 46% (20%–73%). This domain scored the lowest, with a lack of description of facilitators and barriers to guideline implementation.
(6) Editorial independence, 67% (19%–83%). This domain had the widest range among guidelines. While most stated that relevant funding bodies and competing interests had minimal influence on the content of the guideline, some CPGs did not readily offer this information. Graphical representation of the domains is shown in the Figure. Five of 10 guidelines received an overall approval of recommended by at least 4 of the 7 evaluators (Table 1).
|CPG||ICC||95% confidence interval||P value|
|Evaluation and management of obesity hypoventilation syndrome||0.99||0.97||0.99||<.001|
|Preoperative screening and assessment of adult patients with OSA||0.99||0.96||0.99||<.001|
|Diagnosis and treatment of sleep-disordered breathing in adults||0.99||0.97||0.99||<.001|
|Intraoperative management of adult patients with OSA||0.98||0.95||0.99||<.001|
|Perioperative care of patients with OSA undergoing upper airway surgery||0.98||0.95||0.99||<.001|
|Perioperative OSA management in bariatric surgery||0.98||0.92||0.99||<.001|
|Preoperative selection of adult patients with OSA scheduled for ambulatory surgery||0.98||0.93||0.99||<.001|
|Perioperative management of patients with OSA||0.94||0.82||0.99||<.001|
|Anesthetic perioperative care and pain management in weight loss surgery||0.93||0.80||0.99||<.001|
|Diagnosis and treatment of sleep-disordered breathing||0.96||0.87||0.99||<.001|
The practice guideline with the highest overall performance on the AGREE II instrument was published by the ATS.22 This CPG achieved an overall score of 85% across all 6 domains and was recommended by 6 of the 7 evaluators. Three other CPGs achieved an overall score of at least 70%, and were recommended by 5 of the 7 evaluators.23–25
The ICCs of evaluations performed by the 7 evaluators were >0.9 for all included studies (Table 3), which indicated that consistency of the scores among the evaluators was high.
This critical appraisal showed that several CPGs regarding perioperative management of patients with OSA had poor methodological quality, with half of the guidelines not recommended as published. The main items of concern included the domains of stakeholder involvement and applicability, which were lacking alone or in combination in many of the published guidelines.
Clinicians often rely on CPGs to guide decisions regarding patient care. Evidence shows that guidelines with rigorous development processes do improve clinical outcomes.28 Particularly in anesthesiology, in which robust randomized clinical trials are often not feasible to guide “best clinical practice,” the availability and value of high-quality guidelines are most important.29 Consequently, guidelines developed with insufficient rigor can lead to misinformed patient care.
We found that stakeholder involvement regarding the experience and expectations of the target population in development of the guidelines was lacking, a limitation also found in the appraisals of other perioperative guidelines.30–32 Consultation with patients helps ensure that values, preferences, and topics of priority have been considered. There has been an increased shift toward a patient-centered approach in the perioperative care setting that aims to take into account the holistic needs and goals of the patient as a means to improve satisfaction, functional status, and well-being.33,34 Patient and public involvement should be incorporated as a key component of clinical practice guideline development to identify patient-relevant topics (eg, “Am I allowed use the same CPAP (continuous positive airway pressure) mask in the hospital?” “How will the doctors and nurses understand my CPAP machine settings?” and “How will I be able to clean my CPAP machine and mask in the hospital?”), and outcome selection (eg, important elements of patient counseling in the preoperative and postoperative setting). This domain is particularly important to guide patients on care after hospital discharge, when there is ample scope to engage patients in the creation of shared care pathways. Improvements in this domain can make clinical guidelines more accessible to a broader audience.35 Meaningful stakeholder perspectives should be incorporated by future groups developing perioperative guidelines for patients with OSA.
Another domain that scored particularly low was applicability, which concerns facilitators and barriers to implementation of guideline objectives. Although guidelines were developed with rigor, guidelines may still be prone to poor uptake in a clinical setting and have limited improvement on outcomes. Certain facilitators may include providing advice or tools to aid implementation (eg, availability of conducting sleep studies and obtaining reports and addressing CPAP noncompliance), potential resource implications (eg, cost-benefit analysis and application in poorly resourced centers), and the feasibility of monitoring criteria (eg, conducting pilot studies to monitor).36
This study has its limitations. The AGREE II instrument, although a validated tool, requires subjective assessment from evaluators. To control for this, we ensured that all evaluators underwent robust training on using the AGREE II instrument with reference to the user manual during the evaluation period. Each guideline was evaluated by 7 distinct evaluators to reduce the impact of subjectivity. The consistency between evaluators was also demonstrated with strong interrater reliabilities. Our methods only included guidelines that were published in full-text online and in English; therefore, any guidelines that were published in the “gray” literature or were unavailable online or in a different language were omitted. Most guidelines were published by North American societies, although most of the societies involved international authors in their production to provide broader applicability.
Our critical appraisal of CPGs in the perioperative management of OSA found that half of the guidelines are of moderate quality and could be recommended as published. However, the involvement of patient stakeholders and implementation of guidelines into clinical practice were lacking among most of the guidelines. Ongoing collaborative efforts across specialities should focus on guideline development with higher standards utilizing all available quality evidence, wider stakeholder involvement, including a patient perspective, and effective implementation of evidenced-based recommendations to perioperative management.