Authors: Tanaka P et al.
Journal of Clinical Anesthesia, 2025. | DOI: 10.1016/j.jclinane.2025.111922
This study evaluated the reliability and implementation of a mobile web application (Web App) for assessing Entrustable Professional Activities (EPAs) in anesthesiology residency training. Building upon a previously validated set of 14 EPAs, the authors focused on seven EPAs, collecting entrustment data during the 2020–2021 academic year.
A total of 1,116 observations were recorded across 31 residents. Two EPAs demonstrated the ability to discriminate between resident performance levels, while two others failed to show meaningful differentiation. One EPA achieved a generalizability (G) coefficient of 0.68 and phi coefficient of 0.66 with 24 observations, suggesting borderline suitability for formative feedback. However, none of the EPAs reached the 0.7 threshold necessary for moderate reliability.
Qualitative interviews, analyzed through the Consolidated Framework for Implementation Research, identified factors influencing Web App usability, including characteristics of the intervention, users, and organizational environment.
The authors conclude that while not all EPAs may effectively differentiate resident performance, single EPA assessments can still serve as valuable components in Competency-Based Medical Education (CBME) through programmatic assessment strategies. The Web App was positively received for low-stakes, formative assessment purposes.
References:
Authors: Tanaka P et al. 2025.
Journal of Clinical Anesthesia. doi:10.1016/j.jclinane.2025.111922
Thank you to the Journal of Clinical Anesthesia for allowing us to share this article and highlight their journal.