Educational Video-based Scenarios and Evaluative Instrument For Handoff Communication Saba Berhie 1 ; Vineet M Arora 1 ; Leora Horwitz 2 ; Mark Saathoff 1 ; Paul Staisiunas 1 ; Jeanne M. Farnan 1 1 University of Chicago, Chicago, IL; 2 Yale University, New Haven ,CT Background • Single institution and small sample size • Future work planned at partner With generous funding from: • AHRQ R03 Grant # 1R03HS018278-01 • NIA T35 Grant #5T35AG029795-02 Instrument Development: Handoff CEX Communication Professionalism Setting Superior “Check post- transfusion HgB at 2pm. If <10, transfuse 2 U“ No inappropriate comments, not hurried Satisfactor y “the only to do is check CBC at 2pm” “I’ve had a rough day, lets be quick. I’ll start with the sickest first, she’s a train wreck” Un- satisfactor y “Oh yeah, can you check labs?” “the ER could not intubate to save their life” Research Funding Limitations Workshop Objectives & Structure Conclusions • Utilize videos which highlight varying levels of handoff performance. • Rate and assess each video utilizing Handoff CEX instrument WORKSHOP OUTLINE • Two 60-minutes workshops offered • Six videos watched and rated independently by faculty participants using Handoff CEX • Post-workshop roundtable held after viewing the videos • 3-item Handoff CEX shows promise as a reliable and valid tool to assess varying levels of videos depicting handoff performance • Raters had difficulty distinguishing between satisfactory and superior levels • Correlation between communication and professionalism • Future work in real-time clinical scenarios Results • 14 faculty from 2 departments participated • 73 of a possible 90 (82%) handoff observations were captured • Reliability • Cronbach’s alpha: 0.81 • Kendall’s coefficient of concordance: 0.59 • Based on “Mini- CEX” instrument widely used in internal medicine (Norcini,2003) Peer Evaluation Pilot Testing • Internal Medicine residents at the University of Chicago: 366 handoff observations • Factor analysis revealed that three domains captured 82% of the variance of overall sign-out quality • Communication, Professionalism and Setting 0 2 4 6 8 10 Unsatisfactory Satisfactory Superior Communication p <0.001 • Validity • Test of trend across ordered groups: p<0.001 • Two-way ANOVA • No rater bias; results significant only for difference between “superior” & “unsatisfactory” levels of performance • July 2011– ACGME sets new limits for resident duty hours • Unintended consequence is increase in number of handoffs • Few trainees receive education on effective handoff communication (Horwitz, 2006) • Lack of validated instruments to measure handoff performance HOW ACCOMPLISH THIS? Specific Aims • To create video-based examples of varying levels of handoff performance for education, adapting the approach in Arora et al MedEd Portal: “Handoffs: A Typical Day on the Wards” • To validate an assessment instrument: • 6 scripts developed & reviewed • Changing handoff quality in the 3 domains of performance • De-brief instrument development VIDEO DEVELOPMENT MedEd Portal Submission • Arora V, Farnan J, Paro J, Vidyarthi A, Johnson J. Teaching Video:"Handoffs: A Typical Day on the Wards” MedEdPORTAL; 2011 Figure 1: Faculty Ratings of Communication 0 2 4 6 8 10 Unsatisfactory Superior Setting p <0.001 Figure 2: Faculty Ratings of Setting 0 2 4 6 8 10 Unsatisfactory Satisfactory Superior Professionalism p <0.001 Figure 3: Faculty Ratings of Professionalism • Ability to detect level of performance for communication impacted by level of professionalism