e-Portfolio (RCOA): Tips for EdSups/Trainers · the trainee needs to have demonsrated on e-portfolio that the CCLO has been achieved – usually 1 CEX, 1 DOPS, I CbD, plus other supporting

e-Portfolio (RCOA): Tips for EdSups/Trainers

Dr Patrick Mullen

RCOA DRA Anaesthesia, Mersey

24 May 2013, 22Nov13, updated 6th June 2014

Impressions from 4 years of e-Port

• Can demonstrate sufficient evidence for ARCP

• Variability in how people using it

problems with sifting/analysis

• ‘Tips’ consistency ease of use/analysis

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

…RCOA Resources

…RCOA Resources

Lost of useful files for training, queries, etc.

…RCOA Resources

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

…if trainer clicks on assessment tools, then …

Trainer Appraisal Resources

Appraisal: You can print a list of number/type of assessments that you have done, by clicking on the

‘Previously approved/rejected’ tab

Trainer Appraisal: as well as producing this list (or part-printed

for a specific time period), you could also print out one example of each

type of assessment, for your appraisal evidence folder, or look at

this online during your appraisal.

Deanery Resource

www.merseydeanery.nhs.uk/core-anaesthetic-training/portfolio

Will supplement the existing file about e-Port

that is already there

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

pre ARCP

• Upload a ‘CV for ARCP’ (into ‘My Details’)

• Upload ‘logbook summary for whatever grade for the year under review at ARCP

• Upload a ‘Unit of Training sessions summary’ (spread sheet, UOT’s on the left, Sessions 1-20 columns on the right, with dates

• (ST3-4’s: Remember that its ‘20 sessions or equivalent’, and that for some UOT’s you can’t really do sessions, e.g. ST3 Cardio-Resp Arrest, Critical Incidents, etc., so focus on Core Clinical Learning Outcomes)

ARCP Records

This information is updated after ARCP

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

Units of Training list (UOT’s) in Basic Training curriculum

Next 18 months

First 6 months

‘Entries’ = 8 types of Assessments, plus the 2

statements; ‘UOT sign-off’ is an Assessment

+ MSF, GCR (group cons report)

This is an example of a Unit of Training at Basic Training Level: to pass this (competency), the trainee needs to have demonsrated on e-portfolio that the CCLO has been achieved – usually 1 CEX, 1 DOPS, I CbD, plus other supporting evidence (number of sessions/cases, study leave, MSF, GCF, etc.), resulting in an EdSup completion of a ‘UOT sign-off’.

Example: UOT ‘sign-off’ for an ST6

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s (+ ‘Activities’)

• PDP’s

• WPBA’s

• Other/Summary

‘Personal Activities’

(as it appears on Trainee eP screen) (Library)

• = ‘Professional Activities’ that are NOT assessments

• Such as S/L, or Meetings, Audits, etc.

• NOT: EdSup reports (=WPBA, ‘Entries’) PB’s in the Gym!

• MSF’s (=WPBA really, so in ‘Entries’)

• Please REFLECT (=Review) on some (or all) of these

Review = Reflection

Review = Reflection

A more detailed Review (Reflection)

‘Activities’: 1st words & Indexing

• When naming an ‘activity’, choose first word carefully – indexing feature is endemic in e-Portfolio and can be very useful when reviewing ‘items assigned to ARCP’ period

• This means that you should assign items to ARCP regularly, as you go, not last minute.

• E.g. you attend a journal cub ….

What might you enter this as?

1st words

• Journal Club

• Journal Club Colorectal

• Using General/Urol/Gynae Journal Club ST3, will enable all the Activities for this UOT to be grouped together by indexing, allowing for easier review by Supervisor(s)

• For some UOT’s, your EdSup will be r/v’ing your e-Port at 22 months into CT1-2, ST3-4, etc ….so quite a few things to sort through potentially.

(Intermediate Training Curriculum)

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

PDP = Personal Development ‘PLAN’ This is a consultant style PDP: it has a title, a list of

objectives & actions, a time scale, etc. TITLE

Trainee PDP is structured as Plan (=Title), then Objectives below this (with space for Actions, Resources, Measurement)

TITLE

Down here to the left is the button called ‘Add Objective’!

N.B. !!

Your PDP = your Learning Agreement, a list of objectives and actions …for that year. For shorter placements add another objective (specific, e.g. for CardioThor UOT) , still within that ARCP year

Example PDP (Objectives) for CT1 • Pass ‘Initial Assessment of Competency’

• Complete (≥) 1 AUDIT during CT1 year (Title)

• Pass Primary FRCA MCQ exam

• Annual MSF (+/- ‘Group Consultant Feedback’) to assist ARCP etc.

• Progress *Basis of Anaesthetic Practice (0-6 months)

• Progress with *Basic Anaesthesia (6-24 months)

• Pass ARCP

• (* focus on UOT’s or Core Clinical Learning Outcomes)

Title could be: ‘CT1 (Aintree) 2012-13’

Example PDP (Objectives) for ST3 • GENERAL DUTIES sub-units: pro-rata progress during

6m placement at … (specify 2-3 to complete, if poss)

• Paediatrics: ~ 50% completion UOT in DGH 6 months

• Complete (≥) 1 AUDIT during ST3 year (Title)

• Pass/progress final FRCA SAQ/MCQ exam

• Annual MSF (+/- ‘Group Consultant Feedback’) to assist ARCP etc.

• Neuro, Paeds, ICM, CardioThor… can be added as individual objectives into an ST3 or ST4 plan

• Pass ARCP Title could be: ‘ST3 2012-13’, but review progress of each objective at the 3 meetings

with your educational supervisor as you rotate

Actions, Resources, Measurement

• Actions: logbook, WPBA’s, local/regional meetings/courses, e-Learning,

register with RCOA, …

• Resources: S/L, certificates, e-MSF, group consultant feedback, audit dept

• Measurement: you/your EdSup record progress here

• Agree an approach with your EdSup.

• Pad out A R M w’your EdSup (these are your aide memoire) -progress recorded at Mid/End of placement meetings, in ‘Measurement’ box

• Remember S M A R T Objectives/Actions, …

By ST7, your front PDP page might look like…

• ST3 PDP (COCH 6m/LHCH 2w, LWH 3m, RLUH 3m)

• ST4 PDP (UHA 3m, MGH 6m, WCNN 3m)

• ST5 PDP (….)

• ST6 PDP, etc.

• PDP ‘Plans’ (i.e. Titles) remain editable until they are closed (so you can add LWH 3m etc).

• Make sure your EdSup updates (+/- closes off some/most) your objectives before you leave

Areas to talk about: ‘Tips’

• RCOA resources and feedback re Q/I

• Trainer appraisal

• MDSA website e-Port… resources

• ARCP records

• UOT’s

• PDP’s

• WPBA’s

• Other/Summary

WPBA Disparity:

How they are currently seen/used

&

How they could be seen/used

‘Tick Box’

‘Tool Box’

Is this good or bad ‘Quality’?

SITUATION

We’ve all seen these type of ‘completed’ or ‘approved’ WPBA’s

LOTS of variable practice in terms of standards & quality: what is written, where placed, blanks, ….

This narrative ideally better in the ‘Comments’ box below

What competency was being assessed here?

This area is usually well discussed/documented

**Discussing/Putting something Specific into these 2 areas/boxes seems to confound trainers and trainees alike**

ALMAT assigned to: Airway (why?), Orthopaedics, Regional, Sedation, Trauma

= A good/bad quality WPBA??

Direction for further learning going on here,

if a little vague

CT1 WPBA, completed well – critical evaluation & specific, sensible suggestions going on here, with a novice trainee re an IAC competency. Quality of machine checking would not be sufficient to pass the Primary FRCA, but is about right for a novice 7 weeks in.

Critical evaluation, appropriate to CT1 grade.

= A better quality WPBA?

= A good quality WPBA?

Technique

Journals

Drugs

Technique

Technique

Curriculum

Drugs Journals

= A good quality WPBA?

Communication

Good quality WPBA?: Critical evaluation in ‘Improvements’

Sharing of resources/knowledge …..

Also, what’s written likely reflects what happened clinically

Quality

No Critical evaluation in ‘Improvements’ No sharing of resources/knowledge No direction, no real engagement.

Does what’s written reflect what happened?

The lack of any trainer comment or evidence of actual assessment is a stumbling block to demonstrating ‘competency’ for a Unit of Training (UOT)

WPBA Mersey, Review

• < June 2013: E-Port now well up & running

• Trainees/trainers documentation/use of WPBA’s?

• Quality? How to improve?

• Cherry pick lessons/tips for others? - ‘Improvement’ - ‘Further learning and experience’

Proposal: we could use E-Port as an educational resource for trainers? - To solve/share the ‘I cant think of anything to write (discuss) in these 2 areas’.

Methods (Mar-Jun2013)

• WPBA period reviewed: 01Aug-31Dec 2012,

• Not feasible for RCOA to ‘pull’ the data

• (23) trainees: 16 CT1-2 475 WPBA’s 7 ST3-4 132 trainers

• All trainees given choice to opt out (N=0)

Mersey Trainee/Trainers

• 190 Mersey trainees listed on e-Portfolio

• 196 ‘Educational Supervisors’ on e-P

• 152 other ‘Clinical Supervisors’/’Assessors’

475 WPBA’s

~10% of total during the 5m period

(Brain storming)

How many WPBA’s per trainee? (over a 5 month period)

(%) CT ST

Median 21 18

IQR 19-25 12-20

Range 11-37 9-37

CT’ trainees x 16, ST’s x 7

WPBA by type (%)

0

5

10

15

20

25

30

35

40

45

CbD DOPS A-CEX ALMAT

Series1

41

29 28

2

# UOT’s each WPBA assigned to: CT 1-2 (n=365)

0

5

10

15

20

25

30

35

40

0 1 2 3 >3

% 26

36

19

11

8

Low cross-linkage of WPBA’s Many not assigned (>3 months)

‘Assessment’ box completed

• Core trainees: 84%

• Specialist trainees: 83%

• = 1:6 blank (3-5 months later)

Recent change: ‘Assessment’ mandatory (‘Satisfactory/Unsatisfactory/Formative Assessment only’

Interval b/w WPBA’s (days)

0

10

20

30

40

50

60

70

0 50 100 150 200 250 300 350 400

Series1

0

10

20

30

40

50

60

70

80

0 20 40 60 80 100 120

Series1

CT 1-2 (n=365 WPBA)

ST 3-4 (n=110 WPBA)

?

What are people writing in: Areas for improvement

Further learning & experience’

(%) Improvement… Further Learning…

Nothing 54 36

Non-specific entry

28 20

Specific entry

18 44

CT’s only (n=365)

‘Areas for improvement’ ‘Further learning & experience’

(%) Improvement… Further Learning…

Nothing (54) 48 (36) 28

Non-specific (28) 25 (20) 24

Specific (18) 27 (44) 48

(CT’s) in comparison to ST’s

‘Non-Specific’

• (Nothing written) ++++++

• Continued experience in …

• Further experience in …

• Continued familiarisation with …

• Needs more practice in ….

• Needs experience with real people

• Maintaining skills regularly

• More confidence

• We both need updating on this subject!

• Further consideration & practice

• None identified

• Maintaining these skills regularly

• Further experience of such cases as they arise

• Perform more cases

• Encouragement: - keep up good practice - You will get better - Ability to deal with problems will change as new skills are acquired

Specific (= Cherry picking)

e.g.: some trainers are directing trainees to selected websites and other resources, or targeting relevant selected areas for further learning

How many trainees know the case of Elaine Bromiley?

www.daysurgeryuk.net

Or Google ‘BADS’

www.poemsforchildren.co.uk

Paediatrics

How many trainees know why the torch is pointing downwards?!

http://www.aagbi.org/

Lots of cherries!

Go through the various UOT’s

• Then summarise an approach at the end

e.g. CT1: Pre-op Assessment

• Communication: lead WHO, ‘material risk’ (n blks)

• Equipment: calibrating FiO2/EtC02 sensors, testing for leaks, AAGBI www.aagbi.org Guidelines:

• Internet: www.euroscore.org, www.sfar.org, www.nrls.npsa.nhs.uk (correct site surgery) ….

• Special circumstances: communication, sight, etc.

What about what trainees views about WPBA?

• Lit review:

Perspect Med Educ (2013) 2:21-27 Ophthalmology trainees views re obj assessment of OSATS WPBA tool

Mersey Trainees

Survey about WPBA, March 2014

n=66 responders

Q:

Q:

E-portfolio Disparity:

How it is currently seen/used

&

How it could be seen/used

‘Tick Box’

‘Tool Box’

[email protected]

May (+/-) be able to help with any queries

…Healthy discontent is a prelude to progress