In experiential learning, student self-evaluation is used to determine students' self-awareness. At our institution, we ask students to self-assess their performance on Entrustable Professional Activities (EPAs) on Direct Patient Care rotations.
The purpose of this study is to investigate if students can accurately self-evaluate performance on EPAs during patient care practice experiences and if their accuracy and therefore, self-awareness, improves over time.
This is a retrospective study using data that was collected from both student and preceptor final evaluations at three time points: T1 or Direct Patient Care Introductory Pharmacy Practice Experience (IPPE) (second or third year), T2 or the First Direct Patient Care Advanced Pharmacy Practice Experience (APPE) (fourth year), and T3 or the Last Direct Patient Care APPE (fourth year). The primary end point examined change in accuracy over time across all EPAs (global accuracy). The secondary end point examined bias or the under-/overconfidence. The tertiary end point examined if student accuracy for individual EPAs changes over time.
For the primary end point, global accuracy, accuracy significantly improved over time (P < .001) with time point one (T1 = 0.52) being more inaccurate than time point two (T2 = 0.26) and time point three (T3 = 0.027). For the secondary end point, bias (degree of under-/overconfidence), there was no significant difference over time. When examining each individual EPA for changes in accuracy over time, we found individual EPAs showed the same significant trend (P < .001) as global scores.
Students' self-assessment accuracy and therefore self-awareness of their performance on EPAs improves over time. In all EPAs, the significant improvement in accuracy occurs between the students' Direct Patient Care IPPE (T1) and their first Patient Care APPE (T2). This evidence suggests the importance of early exposure of students to direct patient care activities, and that self-evaluation of EPAs is beneficial to student development and leads to improved self-awareness of their performance over time.
CONFLICT OF INTEREST
The authors have nothing to disclose.
- 1, , , . Medical student changes in self-regulated learning during the transition to the clinical environment. BMC Med Educ. 2017; 17(1): 59.
- 2, . The expert learner: Strategic, self-regulated, and reflective. Instruct Sci. 1996; 24(1): 1–24.
- 3, , , . Self-assessment in pharmacy and health science education and professional practice. Am J Pharm Educ. 2010; 74(5): 1–7.
- 4, , . Strategies for improving learner metacognition in health professional education. Am J Pharm Educ. 2017; 81(4): 78.
- 5, . Evaluating the accuracy of pharmacy students' self-assessment skills. Am J Pharm Educ. 2007; 71(5): 89.
- 6, . First-year pharmacy students' self-assessment of communication skills and the impact of video review. Am J Pharm Educ. 2010; 74(5): 1–7.
- 7 Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree: Standards 2016. Chicago, Illinois: Accreditation Council for Pharmacy Education; 2016.
- 8, , , et al. Toward a shared language for competency-based. Med Educ. 2017; 39(6): 582–587.
- 9, , , et al. Transforming primary care training—Patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2013; 28(6): 801–809.
- 10, , , et al. What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-of-training entrustable professional activities for geriatric medicine. J Am Geri Soc. 2014; 62(5): 924–929.
- 11, , , et al. Creation and validation of entrustment scales for the common pediatric subspecialty entrustable professional activities (EPAS). 2016; 16(6): e1-e2.
- 12, , , et al. Pediatric gastroenterology, hepatology, and nutrition entrustable professional activities: Development of an assessment tool and curricular resources. J Ped Gatrointest Nutr. 2020; 71(1): e40–e45.
- 13, , , . Bridging the gap between undergraduate veterinary training and veterinary practice with entrustable professional activities. 2020:e20190051.
- 14, , , . Entrustable professional activities in oral health for primary care providers based on a scoping review. J Dent Educ. 2019; 83(12): 1370–1381.
- 15, , , . Development of undergraduate nursing entrustable professional activities to enhance clinical care and practice. Nurse Educ Today. 2020; 87:104347.
- 16, , , . Building a competency-based workplace curriculum around entrustable professional activities: The case of physician assistant training. Med Teach. 2010; 32(10): e453–e459.
- 17, , , , . Validation of the entrustable professional activities for new pharmacy graduates. Am J Health Syst Pharm. 2018; 75(23): 1922–1929.
- 18, , , , , . Entrustable professional activities for pharmacy practice. Am J Pharm Educ. 2016; 80(4): 57.
- 19, , , , . Developing and implementing an entrustable professional activity assessment for pharmacy practice experiences: A pilot study. Am J Pharm Educ. 2020; 84(11):ajpe7876.
- 20, . Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82(6): 542–547.
- 21, , , , , . Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015; 37(11): 983–1002.
- 22. Trust, competence, and the supervisor's role in postgraduate training. Br Med J. 2006; 333(7571): 748–751.
- 23, , , , . Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Med Teach. 2017; 39(8): 802–807.
- 24. Medical students' self-assessment of performance: Results from three meta-analyses. Patient Educ Counsel. 2011; 84(1): 3–9.
- 25, . Overconfidence produces underachievement: Inaccurate self evaluations undermine students' learning and retention. Learn Instruct. 2012; 22(4): 271–280.
- 26, , . Measuring metacognitive knowledge, monitoring, and control in the pharmacy classroom and experiential settings. Am J Pharm Educ. 2020; 84(5): 549–560.
- 27, , , , . How do supervising physicians decide to entrust residents with unsupervised tasks? A qualitative analysis. J Hosp Med. 2014; 9(3): 169–175.
- 28. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013; 5(1): 157–158.
- 29, . Metacomprehension: A brief history and how to improve its accuracy. Curr Dir Psychol Sci. 2007; 16(4): 228–232.
- 30, , . What constrains the accuracy of metacomprehension judgments? Testing the transfer-appropriate-monitoring and accessibility hypotheses. J Mem Lang. 2005; 52(4): 551–565.
- 31, , . Accurate monitoring leads to effective control and greater learning of patient education materials. J Exp Psychol Appl. 2011; 17(3): 288–302.
- 32, . But I thought I knew that: Using confidence estimation as a debiasing technique to improve classroom performance. Appl Cogn Psychol. 2001; 15(1): 23–32.
- 33, . Effects of grade level and subject on student test score predictions. J Educ Res. 1997; 90(3): 170–174.
- 34, . A comparison of the effects of practice tests and traditional review on performance and calibration. J Exp Educ. 2001; 69(2): 133–151.
- 35, , , . The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. J Exp Educ. 2005; 73(4): 269–290.
- 36, , . Improving metacognition in the classroom through instruction, training, and feedback. Metcog Learn. 2016; 11(2): 215–235.
- 37, , . Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style. Metacog Learn. 2008; 3(2): 101–121.
- 38, , , . Test prediction and performance in a classroom context. J Educ Psychol. 2000; 92(1): 160–170.
- 39, . Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Personal Soc Psychol. 1999; 77(6): 1121–1134.
- 40, . Exploring students' calibration of self reports about study tactics and achievement. Contemp Educ Psychol. 2002; 27(4): 551–572.