Internal and External Assessment of a Successful Quality ... · internal and external metrics. • ACGME Milestone data can track vertical and horizontal change and reflect improving

• Define personal goals/interest/motivation

• Why are you doing this?• Identify key residents & faculty• Who should you work with?• Start small • What is our first goal?

• Discuss “getting started” • Identify benefits of involvement with

resident scholarly activity• Examples of past & current PA

involvement with QI and research

• Milestone data• M&M files• New Innovations• Surveys

• Professional Development • Interactions with residents and faculty• Camaraderie and awareness • ACGME requires scholarly activity• Clinical Research is a GME and

Institutional pillar

Project Goal: Demonstrate the financial and intangible value of surgery residents.

Role: Streamlined report distribution of Resident Duty Hours and Resident Scholarly Activity.

Project Goal: HSQC goal to improve GME QI Education.Role: Created HSQC QI Education Survey, distributed, provided results.

Project Goal: Standardize Trauma Care. Role: Created survey and analyzed survey responses.

Pre Post P-value General Knowledge

I generally understand NSQIP data collection practices. 3.8 4.2 0.09

I understand how to interpret a six-month NSQIP report.

3.4 4.1 0.003

I am familiar with "lean" strategies or "Six Sigma." 2.6 2.9 0.4 I have looked at my personal surgeon report from our NSQIP data and am able to interpret it.

3 3.5 0.2

Attitude I believe that participation in NSQIP and NSQIP tools (risk calculator) is helpful in improving quality surgical care.

4.2 4.5 0.1

I believe that quality improvement training is an important part of resident education.

4.2 4.4 0.2

I believe quality improvement projects improve patient care.

4.2 4.5 0.05

I believe our surgical leadership embraces and follows quality improvement principles and initiatives.

4.1 4.5 0.09

I have the knowledge and skill by which to implement a quality improvement project.

3.8 4.2 0.04

Participation I have participated in a quality improvement project prior to today's date.

4.0 4.8 0.006

Table 1. Results from pre- and post-didactic survey of 25 residents (3 PGY 1, 5 PGY2, 6 PGY3, 4 PGY4, 5 PGY5, 2 PGY 6). Pre-and post-didactic results are compared using Wilcoxon matched-pairs signed rank test, with each resident as their own control. Questions demonstrating improvement are bolded (p<0.05).

Along with clinical outcomes, success of a surgical residency QITI Curriculum can be measured using internal and

external metrics.

Significance StatementMeasuring QI Curriculum

success is critical to adaptation and curriculum improvement.

Lessons Learned• Quantifyingsuccesswith

residentQIeducationisdifficult,butcanbeassessedusinginternalandexternalmetrics.

• ACGMEMilestonedatacantrackverticalandhorizontalchangeandreflectimprovingdepartmentalfamiliaritywithQI.

MM Mrdutt, JL Regner, T Isbell, B Hodges, JS Thomas, HT Papaconstantinou, CL IsbellDepartment of Surgery, Scott & White Medical Center, Temple, TX

Internal and External Assessment of a Successful Quality Improvement Curriculum in Surgery

Scott&WhiteMedicalCenter– Temple

Methods• Objectivetoolstoassessprogresswereincludedinourupdated

QITIcurriculum.• Internalassessmentincludes:pre- andpost-didacticself-

reportedsurveyaddressingknowledge,attitudeandparticipationandQIspecificscholarlyactivity.

• ExternalassessmentincludedACGMESurgicalMilestonePBLI3.

Jan 15 Jan 16 Jan 170

1

2

3

4

Median PBLI3 Class Over Time

PGY 5PGY 4PGY 3

PGY 1PGY 2

PGY 2

PGY 1

PGY 3

PGY 1

Figure. Practice-Based Learning and Improvement (PBLI3) Milestone addressing QI activity and competence. Median score for each PGY class improve over time (Jan 2015-Jan 2017p<0.007). Friedman test compares individual resident scores over time.

Project Goal: Assess success of QITI. Role: Milestone data collection.

Objectives/Goals Surgery Value Analysis

911 Trauma Team

Quality Improvement

Getting Started

Benefits of Involvement

Methods & Tools

HSQC RCA Education

Take Home Points• Identify Allies • Meet, meet again • Set timeline & goals

Leadership Roles- House Staff Quality Council - Program Administrator Liaison RCA

Simulation Work Group- GME Executive Patient & Safety

Committee MemberHave you

participated in an RCA?