Thank you for joining us! Please remember to complete your ...

12/9/2013

1

The Use of an Integrated OSCE to Assess Pharmacy Students’ Clinical SkillsNancy Letassy, Pharm.D., CDE, ProfessorOUHSC College of Pharmacy

Cell phones and pagers should be turned to silent or off. Thank you!

Thank you for joining us! Please remember to complete your pre- and post-evaluations and turn them in at the end of the presentation.

Nancy Letassy, Pharm.D., CDE Professor, OU College of Pharmacy

12/9/2013

2

OSCE Description Limitations of multi-station OSCE Development of an integrated OSCE Test Procedures Advantages and Disadvantages

Objective Structured Clinical Examination

Introduced by Ronald M. Harden, M.D. 1975

University of Dundee, College of Medicine Developed to address

disadvantages of traditional clinical examinations

12/9/2013

3

“An approach to the assessment of clinical competence in which the components of competence are assessed in a planned or structured way with attention being paid to the objectivity of the examination.” RM Harden, MD

Medical Teacher 1988

Medical Teacher 1988;10;19-22

Objective

Assesses competence in a wide range of skills

Uses a rubric to define what is tested and grade assignment

Establishes criteria for passing prior to exam

Employs standard patients

Trains multiple examiners

12/9/2013

4

Structure

Reflects the objectives of the course

Determines in advance exam content and competencies assessed

Assesses a range of competencies through multiple stations

Structures movement through the exam

Organizes stations by time, information, standard patient, and tasks to be performed

Clinical Examination

Assesses clinic performance of skills and application of clinical knowledge

Depends on interaction with a live or simulated patient

Presents “real world” problems, interventions, or tasks to assess or perform

12/9/2013

5

Traditional OSCE Consists of multiple,

timed, standard stations Assesses 1 to 2 specific

clinical tasks per station Conducted in an

interactive environment Uses patient actors or

standardized patients Observed and graded by

trained examiners

History taking Physical

examination Interpretation of

findings Communication,

verbal and written Patient education

12/9/2013

6

Multi-station OSCE limitations in evaluating integrated patient care skills: Assesses a narrow range of skills and knowledge in

isolation

Assesses clinical skills in a number of patients in isolation instead of comprehensively examining a single patient

Assessing isolated parts not equivalent to the whole integrated performance

Limited ability to replicate real-life patient care situations

Some clinical skills cannot be assessed properly

Impact of an absent patient or examiner on exam process

Consider these factors in planning an OSCE Planning and administrative time Number of students to be assessed Space Cost Patient availability Training and cost of standardized patients Set-up time Time to administer exam Personnel required to administer exam

12/9/2013

7

Ambulatory Care Experiential OSCE

Alternate pathway for the Doctor of Pharmacy program

Drug Information Course Ambulatory Care APPE

12/9/2013

8

Fourth Professional Year APPE

9—one month rotations in the P4 year

▪ 2 months of Ambulatory Care

▪ 1 month in acute care

▪ 1 month in community pharmacy practice

▪ 1 month in hospital pharmacy practice

▪ 4 selectives

Family Medicine Center Pharmacist-directed

service Diabetes and

Anticoagulation Service 8 week rotation 4th year Professional

Students see

~100-120 patients/month

12/9/2013

9

Faculty dissatisfaction with traditional assessment methods to evaluate clinical competence

Multi-station OSCE impractical due to space, time, manpower

Integrated OSCE format viewed as a better format to assess clinical skills

OSCE adopted as end-of-the month assessment for APPE in June 2006

Patient Interviews Medication History Taking Interpretation of patient data Identify problems Patient assessment Communication Recommend management Patient education

12/9/2013

10

Patient Encounters (2 videotaped and 1 live) Patient interview Oral communication Professional behavior Medication taking Patient education

SOAP Notes –2 graded Written communication Patient assessment Drug therapy assessment

OSCE

Assess clinical knowledge and skills

Conducted at week 4 and week 8

Five students per month on rotation

Written examination

Assesses knowledge

12/9/2013

11

OSCE status in assessment of students on an ambulatory care APPE

Summative – performance contributes to the student’s course grade

High stakes – defined as an assessment in which poor performance on the assessment could prevent progression in the P4 year.

Grading

Performed by the faculty using a grading rubric

Steps to create an OSCE: Used exam blueprint to develop patient cases Determined skills and knowledge to be

assessed Created realistic patient cases through group

case writing and review Script written for standardized patient

12/9/2013

12

Created grading rubric to assign grades

Determined standard for pass/fail Assigned time for each task Outlined test procedure Trained faculty to serve as patients,

examiners, and providers

Student package includes: Directions Time allotment Physician note Reason for referral Problem list Lab and vitals Past response – INR and dosing history or blood

glucose log Medication refill history Note template

12/9/2013

13

Same as the student package Directions to the standardized patient

History Medications Diet Reasons for adherence problems Disease related Psychosocial factors Directions for what to ask student if they do not ask Directions on what to withhold to make the student

dig for the information

Revisions and maintaining exam security Developed multiple cases Cycle cases 2-3 times throughout the

year Review cases and rubrics prior to each

use Revise cases and rubrics on a periodic

basis

12/9/2013

14

Conducted in the clinic where the students practice throughout the month

OSCE station is a clinic room familiar to the student

All students are examined using the same case on the same day

Assess all clinical competencies at one station

Ambulatory care practice faculty serve as a patient and a provider

12/9/2013

15

Student enters assigned room Given 15 minutes to review patient

information packet and prepare for interview Patient enters room Student is in charge of the visit Given 15 minutes to interview patient, obtain

medication history, determine problem, and educate patient if necessary

Time called and patient exits room

Student has 25 minutes to construct assessment and plan and write SOAP note

Upon exiting room, faculty complete rubric for patient interview

Time is called and faculty enter room as a provider

Student has 5 minutes to present A/P to provider

Provider interaction with student is not scripted

12/9/2013

16

When time is called the faculty collects all material

Faculty exits the station Student’s assessment and plan are graded

according to the rubric by the faculty who played the patient

Student presentation to the provider is graded by the faculty who played the provider

Grading rubric is completed Faculty who played the patient reviews

performance with student Feedback is given to student Student’s are able to provide feedback as well Feedback can be used to revise OSCE cases

as needed

12/9/2013

17

Points

Full

Credit

Partial

Credit

No

Credit

Patient Interview

1 Student introduces him/herself

1 Student addresses patient concerns about treatment regimen

7 Student performs a medication history. The following points are

awarded for all medications with exception of Insulin:

Student performs a medication history using open ended

questions. (1 pt)

The history is complete including all medications and

dosage regimen. (4 pt)

Assess adherence with all medications. (1 pt)

Includes questions about OTC and herbal products. (1 pt)

(specific questions about Insulin are in next line)

4

Student asks patient how she is taking her Lantus and Novolog;

assesses compliance

Students establishes when patient takes insulins and doses

(2 pts)

Assesses adherence to prescribed regimen (2 pt)

Written Assessment/Plan

Full

Credit

Partial

Credit

No

Credit

4

Assessment:

hypoglycemia due to

o decreased food intake ( 1 pt)

o too much Novolog before breakfast (1 pt)

o increased activity during the morning (1 pt)

o atenolol decreasing recognition of

hypoglycemic symptoms (1 pt)

12/9/2013

18

13

Recommendations:

Changes in insulin regimen should include:

o Overall decrease in insulin daily dose –somewhere in

the range of 5 to 10 units.—(1 pt)

o Increase percent of insulin given as basal and reduce

percent given as bolus – towards 50-50%.—(1 pt)

o Specifically decrease Novolog at breakfast when she

eats less or plans to exercise (2 pt)

o F/u with patient in 3 to 7 days to evaluate changes in

blood glucose and hypoglycemia occurrence (2 pt)

Skip Novolog if she does not eat breakfast (2 pt)

Monitoring (4 pt):

o Recheck blood pressure in 2 -4 weeks

o Recheck potassium in 2-4 weeks

o Recheck ALT/AST in 1 -3 months

o Recheck lipids in 1-3 months

Recommends a foot exam to assess sensate, pulses and

vascular status (1 pt)

Full

Credit

Partial

Credit

No

Credit

Presentation to Physician Full

Credit

Partial

Credit

No

Credit

1 Student introduces him/herself and purpose of

presentation

1 Student displayed professional maturity with

presentation

3 Student provides appropriate information from SOAP

note in a complete, concise manner

1 Student convincingly justified his/her plan

3 Student appeared to have utilized the 5 steps of clinical

reasoning (RT modules)

1 Student gives presentation in a professional

manner(appropriate medical terminology, eye contact, no

distracting mannerisms)

10 pt possible

12/9/2013

19

Cost minimal Manpower

One faculty per student tested

One clinic staff to keep time

Minimal test set-up

One hour to prepare exam packets

10-15 minutes for clinic set-up

One hour to administer exam Grading and feedback --30 minutes

Modifications over time Time allowance increased for review of

patient information, patient interview, and time to construct the SOAP note

Grading rubric has become more detailed Instructions for the standardized patient have

become more detailed

12/9/2013

20

Patient case replicates the usual patient-student interaction

All clinical skills are assessed with one patient case

Allows assessment of an integrated clinical performance

Allows an integrated evaluation

Limitations Relator bias of the examiner ▪ Effect of the faculty playing the patient

and provider

▪ Faculty-student relationship

▪ Faculty –patient is also the major evaluator

Only one station

12/9/2013

21

Using an integrated OSCE in APPE assessments Cost, time, and space effective

Minimal personnel requirements

All students see the same case on the same day

Using faculty as standardized patients decreases cost and avoids the problems of no show patients

Allows assessment of clinical skills in an integrated fashion that reflects realistic pharmacist-patient interactions