Implementation of Emergency Checklists: Challenges and Experiences

For over a decade, cognitive aids for critical events in the OR, such as checklists and manuals, have been readily available. Their effectiveness in enhancing the management of rare, high-risk events has been established, albeit in simulation scenarios (Anaesthesia 2023;78:343-55). They serve various functions, including promoting a shared mental model, mitigating cognitive failures under stress, and supplementing memory for infrequently used knowledge, among others. Over the years, valuable lessons have been learned about incorporating these aids into our clinical practices. It is evident that mere display on the wall is insufficient to drive meaningful change.

Indeed, there are striking parallels with the longer history of implementing the World Health Organization Surgical Safety Checklist. Although numerous studies have shown a substantial and consistent reduction in surgical mortality and complications worldwide through checklist adoption, an outlier experience in Ontario, Canada, raised questions about the effectiveness of minimal implementation strategies (N Engl J Med 2014;370:1029-38; N Engl J Med 2014;370:1063-4). In that setting, a top-down government mandate led to introduction of the checklist without local buy-in, and, unlike in other experiences, improvements in patient outcomes were not observed. It becomes clear that implementation is pivotal, particularly in a field that impacts professional culture and hierarchy and requires humility and acknowledgment of human fallibility.

In 2018, experts from Stanford and Harvard conducted an in-depth study on crisis checklist implementation experiences. They identified key steps and utilized their findings from this study to develop an implementation toolkit aimed at supporting adopters (Implement Sci 2018;13:50; asamonitor.pub/3VcFuiR). Specifically, nearly 2,000 survey respondents who had downloaded an OR cognitive aid identified eight crucial implementation tactics (Table).

Many of these steps are instrumental in garnering support and achieving broad, multidisciplinary acceptance of checklists. The study revealed a nearly linear relationship between the number of completed steps and the likelihood of respondents reporting reliable tool usage in practice. Therefore, the implementation toolkit was designed to offer support for each of these crucial steps. Training and retraining pose significant challenges, as effective use does not lend itself to didactic instruction, and access to high-tech simulation training is often limited. Innovative approaches are needed to deliver effective adult learning exercises. Because checklist content is not applicable to all environments, customization has been encouraged but has been technically challenging. Recent updates to the Ariadne Labs (Harvard) crisis checklists have incorporated extensive, user-friendly editing tools to lower the barriers to customizing content to meet local needs (asamonitor.pub/3Vuj0dB).

These insights are invaluable for institution-level implementation. On a larger scale, modification and innovation are imperative for the widespread adoption of these tools. A compelling example can be found in the Chinese experience, as reported by Jeffrey Huang, MD (APSF Newsletter 2016;31:43-5).

Addressing barriers to adoption, such as language and cost, was crucial in spreading the use of cognitive aids. Major OR checklists produced in the United States, such as those from Stanford, Ariadne (Harvard), and the Society for Pediatric Anesthesia, were all translated into Chinese in 2015 (by volunteers!) and made available through the websites of the New Youth Anesthesia Forum, the largest anesthesia network in China, at no cost. This approach proved highly effective, with over 125,000 downloads of the first Chinese version of the Stanford Emergency Manual within the first six months after release (APSF Newsletter 2016;31:43-5).

At an institutional level, presenting cognitive aids at conferences was identified as a critical step. On a national scale, leveraging media platforms can amplify these efforts. For instance, the Chinese New Youth Anesthesia Forum hosted a webinar that drew an audience of over 47,000 (APSF Newsletter 2016;31:43-5). An extensive educational series from the same organization, focused on effective use and covering 22 OR critical events, garnered 130,000 views. An international collaboration with the Chinese American Society of Anesthesiology resulted in a series of online lectures addressing emergency management in obstetrics, which astonishingly garnered more than 2.5 million views (ASA Monitor 2023;87:35-6).

The scale of the Chinese implementation effort demanded an especially aggressive strategy to provide adult, experiential learning experiences for checklist adopters. To drive innovation and expansion, a simulation competition was hosted in the city of Zhongshan, where teams from various institutions demonstrated their simulation training models (APSF Newsletter 2017;32:53-4). The Chinese Association of Anesthesiologists, one of the major professional specialty organizations in China, included a two-hour emergency manual simulation instructor training course in its annual conference (APSF Newsletter 2018;33:60-1). This “train the trainer” approach can be seen as a “force multiplier” that amplifies implementation and training beyond the capacity of an annual meeting.

The acceptance of cognitive aids in clinical practice was bolstered by support from professional anesthesiology organizations. Both the Chinese Association of Anesthesiologists and the Chinese Society of Anesthesiology recommended implementation and recognized the importance of training.

The results to date validate the effectiveness of these innovative implementation tactics. A multi-institutional study in China in 2018 demonstrated a high adoption rate of emergency checklists (Simul Healthc 2018;13:253-60). More than 70% of all respondents reported using emergency manuals during at least one real critical event in the past six months in China. Nearly 90% engaged in self-review or group study of emergency manuals within the same time frame, while almost 70% participated in multidisciplinary simulation training at least once in the past six months. The lessons and insights gained from these activities pave the way for the widespread adoption of tools that are known to improve the management of life-threatening OR emergencies.