Using 360-degree feedback as Using 360-degree feedback as part of Senior Medical part of Senior Medical Performance Review in a public Performance Review in a public hospital settinghospital setting
Dr Dale ThomasDr Dale ThomasDirector Medical Director Medical
ServicesServicesRedcliffe HospitalRedcliffe Hospital
OverviewOverview
Describe the introduction of Describe the introduction of Senior Medical Performance Senior Medical Performance Review including online 360-Review including online 360-degree feedback tooldegree feedback tool
Discuss some pitfalls and Discuss some pitfalls and challenges in how such processes challenges in how such processes could be adopted more broadlycould be adopted more broadly
Performance AppraisalPerformance Appraisal
Well established processes for most Well established processes for most professional groups, including junior professional groups, including junior doctors, but not for senior doctorsdoctors, but not for senior doctors
Assessing trainees vs expert cliniciansAssessing trainees vs expert clinicians Permanent vs temporary employeesPermanent vs temporary employees Maintenance of standards vs Maintenance of standards vs
continuing professional developmentcontinuing professional development
Clinician EngagementClinician Engagement
Senior medical staff are a Senior medical staff are a valuable resource, but valuable resource, but not often given feedback not often given feedback on their performanceon their performance
Doctors are competitive, Doctors are competitive, high achievershigh achievers
Even experts can improve Even experts can improve their performancetheir performance
Senior Medical Senior Medical Performance ReviewPerformance Review Clinical indicators – Clinical indicators –
complications, re-admission complications, re-admission rates, length of stayrates, length of stay
Participation in CME/CPD/MOPSParticipation in CME/CPD/MOPS Participation in departmental Participation in departmental
meetings & clinical auditmeetings & clinical audit Records of mandatory trainingRecords of mandatory training 360 degree feedback summary360 degree feedback summary
360 degree feedback 360 degree feedback tooltool Based on a validated questionnaire Based on a validated questionnaire 11
Developed following extensive Developed following extensive consultation with AMA and unions consultation with AMA and unions 22
Anonymous online tool or paper surveyAnonymous online tool or paper survey
1. Ramsey PG, Wenrich MD, Carline JC, et al, Use of peer ratings to evaluate physician performance, JAMA 1993 Apr 7;269(13):1655-1660
2. Johnston A, Senior medical performance review: making it happen – the Queensland experience, RACMA Quarterly 2011 Feb, accessed online http://www.racma.edu.au/index.php?option=com_content&view=article&id=301:senior-medical-performance-review-making-it-happen-the-queensland-experience&catid=21:the-quarterly-february-2011&Itemid=14
360 degree feedback 360 degree feedback tooltool 9-point Likert rating scale9-point Likert rating scale
– Clinical domain 9 questionsClinical domain 9 questions– Team management 4 questionsTeam management 4 questions– Interpersonal skills 5 questionsInterpersonal skills 5 questions– Ethical behaviour 4 questionsEthical behaviour 4 questions
Free text strengths & weaknessesFree text strengths & weaknesses http://www.surveymonkey.com/s/SY3BTS7
Implementation at Implementation at Redcliffe HospitalRedcliffe Hospital
Pilot study in Anaesthetic Pilot study in Anaesthetic Department Department – First cohort of 5 anaesthetists in 2010First cohort of 5 anaesthetists in 2010– Second cohort of 6 consultants in 2011Second cohort of 6 consultants in 2011– Third cohort of 5 anaesthetists in 2013Third cohort of 5 anaesthetists in 2013
Physician cohort of 6 consultants in Physician cohort of 6 consultants in 20122012
Stratified design allowing subgroup Stratified design allowing subgroup analysesanalyses
Sampling methodology Sampling methodology 11 Allowing subjects to choose who Allowing subjects to choose who
to invite to provide feedbackto invite to provide feedback Completed by consultant peers, Completed by consultant peers,
supervisors, training registrars supervisors, training registrars and nursing/support staffand nursing/support staff
Collated feedback provided to Collated feedback provided to SMO/VMO for discussion with their SMO/VMO for discussion with their Clinical DirectorClinical Director
Sampling methodology Sampling methodology 22 Consistent sample for each subjectConsistent sample for each subject RespondentsRespondents
– Supervisor/line managerSupervisor/line manager– Consultant anaesthetistsConsultant anaesthetists– Anaesthetic registrarsAnaesthetic registrars– Nursing and allied health staffNursing and allied health staff– Proceduralists – i.e. consultant Proceduralists – i.e. consultant
surgeons, obstetricians, surgeons, obstetricians, gastroenterologistsgastroenterologists
– Optional self-assessmentOptional self-assessment
Response RateResponse Rate
Managers 94%Managers 94% Peer consultants 74%Peer consultants 74% Registrar trainees 57%Registrar trainees 57% Nursing/support 76%Nursing/support 76% Proceduralists 40%Proceduralists 40% Self-assessment 81%Self-assessment 81% Total excluding self 60%Total excluding self 60%
0
12
34
5
67
89
10
OVERALL C
LINIC
AL SKIL
L
COMPLE
X PROBLE
M M
GT
INPATIE
NT MGT
OUTPATIENT M
GT
MEDIC
AL KNOW
LEDGE
PROCEDURAL SKIL
LS
PSYCHOSOCIAL
ASPECTS
QUALITY A
CTIVIT
IES
OVERALL C
OMPETENCE
ACCESSIBIL
ITY
EDUCATION
PROBLEM
SOLV
ING
CULTURE
INTEGRIT
Y
RESPECT FOR O
THERS
COMM
UNICATIO
N
ACCEPTANCE OF C
RITIC
ISM
INTERPERSONAL
RELATIO
NSHIPS
COMPASSIO
N
RESPONSIBIL
ITY
PUNCTUALITY A
ND RELI
ABILIT
Y
SELF M
ANAGEMENT
ResultsResults
Higher response rate for subjects Higher response rate for subjects who are Staff Specialists (72%) who are Staff Specialists (72%) compared with VMO/SMO (44%)compared with VMO/SMO (44%)
Overall very good quantitative Overall very good quantitative scores from all groups of scores from all groups of respondentsrespondents
QualitativeQualitative
75.5% of respondents provided 75.5% of respondents provided commentscomments
63% of comments were positive63% of comments were positive Negative comments can be Negative comments can be
confronting, but were considered confronting, but were considered more useful by subjectsmore useful by subjects
Feedback to subjectsFeedback to subjects
Quantitative & qualitative resultsQuantitative & qualitative results Summary data included as part of Summary data included as part of
annual PAD processannual PAD process Inform professional development Inform professional development
needs for coming yearneeds for coming year
Pitfalls & challengesPitfalls & challenges
Choosing your respondentsChoosing your respondents Online confidentiality concernsOnline confidentiality concerns Lower response rates for VMO Lower response rates for VMO
and non-specialist SMOand non-specialist SMO Meaningful subgroup analysesMeaningful subgroup analyses Evaluating effectivenessEvaluating effectiveness
Questions?Questions?