Post-Graduate Psychiatry Education Evaluation: … Post-Graduate Psychiatry Education Evaluation: Singapore’s experience Associate Professor John CM Wong Vice Chairman Medical Board

ResearchResearch

Post-Graduate Psychiatry Education Evaluation: Singapore’s experience

Associate Professor John CM Wong Vice Chairman Medical Board (Education), National University Hospital (NUH)Head, Dept of Psychological Medicine,National University of Singapore and NUH

14 Sept 2011

Clinical careClinical care

EducationEducation

Scope

Overview of psychiatry training programs 1960s till nowNew approaches to psychiatry training in Singapore

Curriculum evaluationTrainee evaluationFaculty teaching evaluation

46 Years of Transformational Journey: A Singapore’s Experience

An Artificial DiamondCreated

ByStress

SINGAPORE

The City State of Singapore

5

•Population = 4.6m (3.6m residents)

•2007 GDP = S$225B (US$150B)

•Area = 704 sq km (22% reclaimed)

Necessity Compelled Singapore to Create Space

“What we lack in numbers,

we will make up for in

quality: in the standards of

discipline, dedication and

leadership …”Minister MentorLee Kuan Yew

A Small Country Can Survive and Thrive

Agile

Integrated

•Decide Quickly•Implement Quickly•Change Course Quickly

•Within Organizations•Across Organizations•Over Time

Open

•Open to Talent•Open to Investment•Open to New Ideas

•Instinct for Survival•Helpful & Relevant

To Achieve Happiness, Prosperity and Progress for To Achieve Happiness, Prosperity and Progress for Our NationOur Nation

1. Overview of Postgraduate Psychiatry Training in Singapore

Evolution of Psychiatry training in Singapore: 3 Phases

1. 1960s to 1980: Local and United Kingdom (UK Royal College of Psychiatrists)

2. 1980 to 2010: 3 years Basic Specialty Training and 3 years Advanced Specialty (6 years)

3. 2010 May onwards: 5 year Psychiatry Residency Program (ACGME-International)

1980s- 2000: Focus of Specialty Training

3 Basic areas of training:KnowledgeSkills Attitude

‘Apprenticeship’ and mentoringIndividualized In-depth trainingSummative Assessment and Clinical examinations Problem with capacity and consistency (lacking standardization)

Over at NUH & NUS

Our VisionShaping Medicine for the Future

Our MissionAdvancing Health by Integrating Excellent

Clinical Care, Research and Education

Students today

Clinical Care

Research

Teaching

At National University Hospital, At National University Hospital, Medical Education is important in Medical Education is important in achieving achieving ……....

Medical Education EnterpriseMedical Education Enterprise

Teaching Focus Teaching Focus

1. Basic Specialty Training (BST) Programme• Trainees are selected by Specialist Training Committee

for a particular discipline.• During BST, trainee acquire a postgraduate

qualification, M.Med of the National University (NUS) or its equivalent in the relevant specialty.

Basic Traineeship SpecialtyAnaesthesiology OtorhinolaryngologyDiagnostic Radiology Paediatric MedicineEmergency Medicine PathologyInternal Medicine PsychiatryObstetrics & Gynaecology Public HealthOccupational Medicine SurgeryOphthalmology Radiation Oncology

Postgraduate TrainingPostgraduate Training

2. Advanced Specialty Training (AST) Programme

• Trainees are selected by the Joint Committee on Specialist Training (JCST).

• Upon certified of satisfactory completion of AST programme, trainee apply for Specialist Registration with the Specialist Accreditation Board.

• Trainee eligible for promotion to Associate Consultant

Postgraduate TrainingPostgraduate Training

Anaesthesiology Infectious Diseases Paediatric SurgeryCardiology Internal Medicine PathologyCardiothoracic Surgery Medical Oncology Plastic SurgeryDermatology Neurology PsychiatryDiagnostic Radiology Neurosurgery Public HealthEmergency Medicine Nuclear Med Radiation OncologyEndocrinology Obstetrics & Gynaecology Rehabilitation MedicineGastroenterology Occupational Medicine Renal MedicineGeneral Surgery Ophthalmology Respiratory MedicineGeriatric Medicine Orthopaedic Surgery RheumatologyHaematology Otorhinolaryngology UrologyHand Surgery Paediatric Medicine

Advanced Traineeship Specialty

Postgraduate TrainingPostgraduate Training

Senior Consultants

Consultants

Associate Consultants

Registrars / Residents

Medical Officers / Residents

House Officers / Residents

Specialists

Advanced Specialty Trainees / Residency

Basic Specialty Trainees / Residency

Residency

Clinical Career ProgressionClinical Career Progression

2. Program Assessment of Basic & Advanced Psychiatry Specialty Training Program

Performance Standards for PG Teaching – Global Ratings

Minimum Standard : MoH funding guidelines

50% : 1) 4 hrs of protected training time per week; 2) 12 days of study leave per year; and

50% : 3) a minimum qualitative rating of 7 and above on a rating scale of 10 by trainees and the STCs.

5 Questions for Assessment of PostingStructure & Programme1) Is there protected time made available weekly for your training? Yes / No

2) Are there structured training programmes provided in the department that is relevant to your training? Yes / No

Interaction and Supervisor’s Report3) Are the arrangements for formal meetings with designated supervisors adequate?

(Recommended frequency: Once per month) Yes / No

Cases & Procedures4) Are the range of cases / procedures that you managed / performed useful and relevant to your specialty? Yes / No

Overview5) Globally, how would you rate the department that you are posted to in terms of its usefulness and effectiveness in your training as a trainee

(Please select)Scale of 1 – 10 (1 to 4 – Below Average, 5 to 6 – Average, Above 7 – Good)

Postgraduate Survey Questions

Performance ThresholdPG Trainees’ Rating of Training

(7 to 10) for Nov 09 – Oct 10

Perfect Scores: • Lab Med• Gastro&Hepato,• Neonates• Paeds Surgery• Pathology• Urology

Most Improved: • Gastro&Hepato• Paeds Surgery

Large Depts (28 trainees):• Paeds (96%)

PG Training Deliverables 4 hrs of Protected Training Time / Week

3. New Psychiatry Residency Program

Institutional structure that will accord adequate resources for training and education

Training focus on 6 Core Competencies

Planned learning experiences for trainees so that they see the range and number of cases needed to develop necessary competencies

Designated core faculty with protected time to plan curriculum, guide and supervise Residents, perform formative assessments, and

Regular structured formative assessment that is inbuilt to provide continuous, periodic assessment of residents’ skills and performances with multiple evaluations of institution, program, faculty and residents.

Accreditation by the ACGME - Accreditation Council of Graduate Medical Education (US based)

6 Core Competencies: Residency

Competence

Professional medical competence : “the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflection in daily practice, for the benefit of individual and community being served”Incorporates cognitive, technical, integrative, contextual, relational, affective, moral and mindful qualities

National University Health Systems SingHealth Tan Tock Seng

Hospital

3 Sponsoring Institutions (SI)3 Sponsoring Institutions (SI)

Our VisionOur Vision

Shaping future medical leaders

Our MissionOur Mission

Providing holistic medical training to develop evidence-based and compassionate clinicians,

in a vibrant academic environment and a culture of continual improvement

NUHS Residency NUHS Residency –– Vision and MissionVision and Mission

• Careful selection of residents based on alignment to the NUHS values of Teamwork, Respect, Integrity, Compassion and Excellence

• Close structured training and formative assessment & evaluation

• Strive to have the best training program in each specialty and to ensure that our graduates are highly competent and sought after

CEO and CMB leading our residents on a 3 day 2 night Outward Bound Singapore Orientation!

Core Education Program that takes place every Friday 2pm to 4pm

The NUHS Residency Program The NUHS Residency Program ––Shaping our future medical leadersShaping our future medical leaders

Resident Evaluations

Methods of evaluation Core competencies

1 2 3 4 5 6

Record review x x xChart stimulation recall x x x x xCheck list x x x x xGlobal rating x xStandardized patient x x xObjective structured clinical evaluation (OSCE) x x x x xStimulations and models x x x x3600 global rating x x x x x xPortfolios x x x xMultiple choice questions exam (MCQ exam) x x xOral exam x x x xProcedures or case logs x x xPatient survey x x x x x x

Model for therapeutic risk (suicide) management: integrated into core competencies trainingRL Frierson et al 2009

Patient CareResident ask all new patients suicidal ideation

& history past attemptDisplay compassion, establish therapeutic

allianceDevelop systematized treatment plan on

patient's needs

Medical knowledgeResident familiar with risk factors for suicide Understand appropriate treatment modalities for

psychiatric illnesses associated with suicidal behavior

Familiar with emergency retention & mental health act statutes

Practice based learningResident consults supervisor when treating

patients with identified increased suicide riskFamiliar with phenomenology of all suicidal

behavior that impact on managementResidents participate in Suicide Death Review

Round

Interpersonal & Communication SkillsResidents attuned to attitudes about suicidal

behavior in patient’s cultural contextPresent the need for involuntary hospitalization

to suicidal patientsCommunicates to supervisor on patient's

clinical progress

ProfessionalismResident seeks patient’s permission to involve

family members in treatment and discharge planning

Manage medical legal reporting under supervision

System-based practicesResident documents a comprehensive risk

assessment including case specific considerations

Awareness of patient placement options under psychiatric emergencies

Collaborates with psychologists, nurses, social workers in mutli-disciplinary care plan

Resident Performance Evaluation and Feedback Flow-Chart

Daily Informal Guidance by Physician Faculty

Mode of Evaluation

Domains and ToolsOf Evaluation

FeedbackProcess

6 Core Competencies.Direct Observation

Mini-CEXCase-based Discussion

Learning EvaluationChart / Patient Survey

Direct constructive feedback and reinforcement of desired skills& attributes.

Timely correction of performance issues.

End of Assignment by Subspecialty Faculty

(Monthly)

6 Core Competencies.End-Rotation (NI).

MCQ and/or Viva Test.Case-based Discussion.

Results of Resident Evaluation to be done in confidential manner

PC will collate, de-identify and summarizemonthly resident evaluations for CCC review.Results of MCQ and CbD to Training Portfolio.

3 monthly Mentor – Resident Feedback

(Sep, Dec, Mar, Jun)

Fulfillment of Clinical & Academic Goals.

Identification of Training & Case-load Shortfalls.

Identification of Impediments to Education & Training.

Direct constructive feedback on Resident clinical and academic performance.

Reinforcement of good and excellent performances.

Faculty with Resident can escalate system or curriculum based issues to PD and APD to improve the next round of training

rotations.

6-monthly CCC Review(Dec, Jun)

6 Core Competencies.Collated End-Rotation Evaluations

(NI).Collated DOPS, Mini-CEX, CbD,

MSF reports.Collated MCQ, Viva, ITE, MMed

Examinations.

Results of Evaluation in confidential mannerPC will collate, de-identify and summarize

resident evaluations for CCC review.CCC makes recommendations in Training Portfolio.

6-monthly PD Review with Resident(Jan, July)

CCC Review Report.Identify training shortfalls.Reward excellent / good

performances.

Identify the Resident with difficulties with appropriate remedies.Identify Resident with high potential to ‘push’ them further.

Formulate short term milestones and long term career goals with the Resident.

Resident Evaluation

360 –degree assessmentDirect Observation (mini CEX)Standard written and oral examinationsStandardized clinical examinationsPortfolios

4 months 4 months 4 months

Combined Comm Psy PsyG (IMH, CGH) Addiction (IMH)

Addiction (IMH) Combined Comm Psy PsyG (IMH, CGH)

PsyG (IMH, CGH) Addiction (IMH) Combined Comm Psy

Continuity Clinics / Electives Programme every afternoon*

MMED Part 1

12 months

Electives in (1) General Psychiatry, (2) Forensic Psychiatry, (3) Addiction Medicine, (4) Geriatric Psychiatry, (5) Child and Adolescent Psychiatry, (6) Community Psychiatry, (7) EPIP, (8) Research, (9) Consultant Liaison Psychiatry (including Eating Disorder, Perinatal Psychiatry, Neuropsychiatry), (10) Military Psychiatry, (11) or other RRC approved rotations

Exit

R2

R3

R4

R5

R1 6 months 6 months

Neurology Gen Medicine Gen Psy – inpatient (IMH, TTSH, CGH, SGH, NUH)

*Didactics one session per week / Introductory Psychotherapy Lectures

National Psychiatry Residency Programme

4 months 4 months 4 months

Emerg (IMH) CL (TTSH, KTPH, CGH, SGH, NUH) Forensic (IMH)

*Didactics one session per week

Forensic (IMH) Emerg (IMH) CL (TTSH, KTPH, CGH, SGH, NUH)

Child (IMH, NUH) Elective Gen Psy – inpatient (IMH, TTSH, CGH, SGH, NUH) MMED Part 2

4 months 2 months 4 months 2 months

Gen Psy – inpatient (IMH, TTSH, CGH, SGH, NUH) Child (IMH, NUH) Elective MRC Psych

Continuity Clinics / Psychotherapy every afternoon* Part 1, 2, 3

*Didactics one session per week CACS

*Didactics one session per week

Surgery (4 months) Gen Medicine (4 months) Neurology Gen Psy (IMH, TTSH, CGH, SGH, NUH)

CL (TTSH, KTPH, CGH, SGH, NUH) Forensic (IMH) Emerg (IMH)

May July Sep Nov Jan Mar May Jul

Faculty Requirements

Physician Faculty to Resident RatioIn addition to the program director, the core physician faculty to resident ratio must be no less than 1:6.The ratio of all physician faculty to residents, which includes all core faculty and the program director, should be 1:1.

Structured Portfolio

Medical record audit and QI project

1 / yearPatient

+Nurse or peer surveys:

2xyear

Practice-based learning and improvement

Systems-based practice

*Mini-CEX:

4-6/year

Interpersonal skills and Communication

I*TE:1/yea

r

Patient care

*Faculty Evaluations

EBM/Question

Log

Medical knowledge

Professionalism

University Hospital Medical Centre - 2012

National University of SingaporeSchool of Medicine’s

Centre for Translational Medicine MD6

Simulated OR

Simulated ICUs

Simulated Ward Clinical Skills

Centre for Translational Medicine Stimulated Hospital

Beyond post graduate psychiatry training: Maintenance of certification

Current: Singapore Medical CouncilValid qualifying accredited trainingEvidence of participation in continued medical education (CME) through academic, clinical training and scholastic programs

Future: 4 basic componentsEvidence of:

Professional standingCommitment to lifelong learning & periodic self assessmentCognitive expertiseEvaluation of performance in practice