Using 360-degree feedback as part of Senior Medical Performance Review in a public hospital setting Dr Dale Thomas Director Medical Services Redcliffe.

Using 360-degree feedback as Using 360-degree feedback as part of Senior Medical part of Senior Medical Performance Review in a public Performance Review in a public hospital settinghospital setting

Dr Dale ThomasDr Dale ThomasDirector Medical Director Medical

ServicesServicesRedcliffe HospitalRedcliffe Hospital

OverviewOverview

Describe the introduction of Describe the introduction of Senior Medical Performance Senior Medical Performance Review including online 360-Review including online 360-degree feedback tooldegree feedback tool

Discuss some pitfalls and Discuss some pitfalls and challenges in how such processes challenges in how such processes could be adopted more broadlycould be adopted more broadly

Performance AppraisalPerformance Appraisal

Well established processes for most Well established processes for most professional groups, including junior professional groups, including junior doctors, but not for senior doctorsdoctors, but not for senior doctors

Assessing trainees vs expert cliniciansAssessing trainees vs expert clinicians Permanent vs temporary employeesPermanent vs temporary employees Maintenance of standards vs Maintenance of standards vs

continuing professional developmentcontinuing professional development

Clinician EngagementClinician Engagement

Senior medical staff are a Senior medical staff are a valuable resource, but valuable resource, but not often given feedback not often given feedback on their performanceon their performance

Doctors are competitive, Doctors are competitive, high achievershigh achievers

Even experts can improve Even experts can improve their performancetheir performance

Senior Medical Senior Medical Performance ReviewPerformance Review Clinical indicators – Clinical indicators –

complications, re-admission complications, re-admission rates, length of stayrates, length of stay

Participation in CME/CPD/MOPSParticipation in CME/CPD/MOPS Participation in departmental Participation in departmental

meetings & clinical auditmeetings & clinical audit Records of mandatory trainingRecords of mandatory training 360 degree feedback summary360 degree feedback summary

360 degree feedback 360 degree feedback tooltool Based on a validated questionnaire Based on a validated questionnaire 11

Developed following extensive Developed following extensive consultation with AMA and unions consultation with AMA and unions 22

Anonymous online tool or paper surveyAnonymous online tool or paper survey

1. Ramsey PG, Wenrich MD, Carline JC, et al, Use of peer ratings to evaluate physician performance, JAMA 1993 Apr 7;269(13):1655-1660

2. Johnston A, Senior medical performance review: making it happen – the Queensland experience, RACMA Quarterly 2011 Feb, accessed online http://www.racma.edu.au/index.php?option=com_content&view=article&id=301:senior-medical-performance-review-making-it-happen-the-queensland-experience&catid=21:the-quarterly-february-2011&Itemid=14

360 degree feedback 360 degree feedback tooltool 9-point Likert rating scale9-point Likert rating scale

– Clinical domain 9 questionsClinical domain 9 questions– Team management 4 questionsTeam management 4 questions– Interpersonal skills 5 questionsInterpersonal skills 5 questions– Ethical behaviour 4 questionsEthical behaviour 4 questions

Free text strengths & weaknessesFree text strengths & weaknesses http://www.surveymonkey.com/s/SY3BTS7

Implementation at Implementation at Redcliffe HospitalRedcliffe Hospital

Pilot study in Anaesthetic Pilot study in Anaesthetic Department Department – First cohort of 5 anaesthetists in 2010First cohort of 5 anaesthetists in 2010– Second cohort of 6 consultants in 2011Second cohort of 6 consultants in 2011– Third cohort of 5 anaesthetists in 2013Third cohort of 5 anaesthetists in 2013

Physician cohort of 6 consultants in Physician cohort of 6 consultants in 20122012

Stratified design allowing subgroup Stratified design allowing subgroup analysesanalyses

Sampling methodology Sampling methodology 11 Allowing subjects to choose who Allowing subjects to choose who

to invite to provide feedbackto invite to provide feedback Completed by consultant peers, Completed by consultant peers,

supervisors, training registrars supervisors, training registrars and nursing/support staffand nursing/support staff

Collated feedback provided to Collated feedback provided to SMO/VMO for discussion with their SMO/VMO for discussion with their Clinical DirectorClinical Director

Sampling methodology Sampling methodology 22 Consistent sample for each subjectConsistent sample for each subject RespondentsRespondents

– Supervisor/line managerSupervisor/line manager– Consultant anaesthetistsConsultant anaesthetists– Anaesthetic registrarsAnaesthetic registrars– Nursing and allied health staffNursing and allied health staff– Proceduralists – i.e. consultant Proceduralists – i.e. consultant

surgeons, obstetricians, surgeons, obstetricians, gastroenterologistsgastroenterologists

– Optional self-assessmentOptional self-assessment

Response RateResponse Rate

Managers 94%Managers 94% Peer consultants 74%Peer consultants 74% Registrar trainees 57%Registrar trainees 57% Nursing/support 76%Nursing/support 76% Proceduralists 40%Proceduralists 40% Self-assessment 81%Self-assessment 81% Total excluding self 60%Total excluding self 60%

0

12

34

5

67

89

10

OVERALL C

LINIC

AL SKIL

L

COMPLE

X PROBLE

M M

GT

INPATIE

NT MGT

OUTPATIENT M

GT

MEDIC

AL KNOW

LEDGE

PROCEDURAL SKIL

LS

PSYCHOSOCIAL

ASPECTS

QUALITY A

CTIVIT

IES

OVERALL C

OMPETENCE

ACCESSIBIL

ITY

EDUCATION

PROBLEM

SOLV

ING

CULTURE

INTEGRIT

Y

RESPECT FOR O

THERS

COMM

UNICATIO

N

ACCEPTANCE OF C

RITIC

ISM

INTERPERSONAL

RELATIO

NSHIPS

COMPASSIO

N

RESPONSIBIL

ITY

PUNCTUALITY A

ND RELI

ABILIT

Y

SELF M

ANAGEMENT

ResultsResults

Higher response rate for subjects Higher response rate for subjects who are Staff Specialists (72%) who are Staff Specialists (72%) compared with VMO/SMO (44%)compared with VMO/SMO (44%)

Overall very good quantitative Overall very good quantitative scores from all groups of scores from all groups of respondentsrespondents

QualitativeQualitative

75.5% of respondents provided 75.5% of respondents provided commentscomments

63% of comments were positive63% of comments were positive Negative comments can be Negative comments can be

confronting, but were considered confronting, but were considered more useful by subjectsmore useful by subjects

Feedback to subjectsFeedback to subjects

Quantitative & qualitative resultsQuantitative & qualitative results Summary data included as part of Summary data included as part of

annual PAD processannual PAD process Inform professional development Inform professional development

needs for coming yearneeds for coming year

Pitfalls & challengesPitfalls & challenges

Choosing your respondentsChoosing your respondents Online confidentiality concernsOnline confidentiality concerns Lower response rates for VMO Lower response rates for VMO

and non-specialist SMOand non-specialist SMO Meaningful subgroup analysesMeaningful subgroup analyses Evaluating effectivenessEvaluating effectiveness

Questions?Questions?