® Scoring the Clinical Proficiency Examination Dr. Beth Sabin, Assistant Director AVMA Education and Research Division CLEAR Annual Conference September.

®

Scoring the Clinical Proficiency

Examination

Dr. Beth Sabin, Assistant DirectorAVMA Education and Research Division

CLEAR Annual ConferenceSeptember 16, 2005

®

Clinical Proficiency Examination

3.5-day, 7-section, hands-on assessment of clinical veterinary medical skillsMust pass all sections

Administered at multiple ECFVG-approved sites according to Manual of Administration

Also administered by the Canadian National Examining Board (NEB)

®

Educational Commission for Foreign

Veterinary Graduates

Seven-member AVMA committee Oversees educational equivalency

assessment program Candidates are graduates of international

non–AVMA-accredited veterinary schools Certificate holders meet educational

prerequisite for licensure or employment

®

ECFVG Program Steps

1. Enroll; proof of graduation2. Assessment of English language

competency (ETS, CAEL, IELTS)3. Assessment of basic and clinical

sciences knowledge (NAVLE)4. Assessment of hands on clinical skills

(CPE, postgraduate ECY)

®

Clinical Proficiency Examination

3.5-day, 7-section, hands-on assessment of clinical veterinary medical skillsAnesthesiologyClinical and Anatomic PathologyEquine PracticeFood Animal PracticeRadiologySmall Animal PracticeSurgery

®

Clinical Proficiency Skills

CPE Manual of Administration defines as:“…combination of knowledge and technical

skills.”

Examiner assesses knowledge by directly observing examinee behaviors as examinee performs basic skills required of a minimally competent entry-level practitioner.

®

CPE Audit

Audit conducted in Spring 2001 Findings provided to ECFVG Fall 2001

CPE accomplishes the intended goal - it assesses the minimal clinical competency of a graduate of a non-accredited veterinary program

Suggestions for enhancements

®

Post-audit CPE Activities

Examiners’ Training Review of CPE Manual of Administration Examiners’ Master Manual Review of CPE scoring parameters Survey of faculty and students

®

CPE History

Experts selected most pertinent skills to evaluate performance competency

Scoring process defined Goal is to separate “masters” from “non-

masters” Interested in evaluating for minimal

competency

®

Performance Benchmarks

Consider the behaviors to be observed and what constitutes minimal levels of performance at each CPE scoring level:SuperiorGoodSatisfactoryUnsatisfactory

®

CPE Rating Scale(for six of seven sections)

100% - 86% Superior Superior performance of required clinical skills

85% - 71% Good Good performance of required clinical skills

70% - 61% Satisfactory Satisfactory performance of required clinical skills

< 60% Unsatisfactory Failed to demonstrate the required clinical skills

®

Development of Assessment

Methods Competencies identified for each section Example—Clinical and Anatomic Pathology

Recognize and interpret visual images of gross pathological lesions

Perform a necropsy and take tissue samples for histopathology

Perform common laboratory tests and interpret results of those tests in the areas of clinical microbiology, clinical parasitology, hematology, and urinalysis

®

Assessment of Skills

Skills/activities identified for each competency

Example—Perform necropsy Thoroughly examine intact carcass Opens carcass, examine major cavities, remove

viscera and head Examine major organs Examine muscles, joints, bone marrowExamine endocrine glands, lymph nodes

®

Behavioral Anchors

Observable behaviors identified for each skill/activity

Example—Thoroughly examine intact carcass (5 pts)

1 pt: Assesses general body condition/hydration 1 pt: Examines skin/hair coat 1 pt: Examines head 1 pt: Examines limbs including feet 1 pt: Examines perianal and genital region, mammary

glands

®

Calculating Section Scores

Determine weighting of each competency Example—Clinical and Anatomic Pathology

Recognize and interpret visual images: 16%Perform necropsy: 20%Perform lab tests and interpret results: 64%

16% for each of four areas

Sum weighted station scores = section score Pass = 60

®

CPE Rating Scale(for surgery)

2004 review by subject matter experts and testing consultants

Minimal competency best assessed via pass/fail

Experts developed:Check list of passing behaviorsCheck list of failing behaviorsAppendix of fatal flaws

®

Scoring Surgery

CPE Manual states: A “pass” = at least a minimally competent

performance of entry-level surgical skills Examiner to use the surgical skills

assessment sheet as a checklist and to record positive and negative behaviors

Only feedback provided is in the case of a fatal flaw

®

Surgery Competencies

Prepare the patientPrepare selfPerform surgical procedure

®

Surgical Skills

AsepsisOvariohysterectomy technique Tissue trauma Hemostasis

®

Observable Behaviors

Tissue trauma Passing behaviors

Demonstrates atraumatic tissue handling throughoutSkills do not result in cumulative traumatic handling of

tissue Failing behaviors

Causes significant damage to other tissues/organs (fatal)

Skills result in cumulative traumatic handling of tissue

®

Failing Surgery

CPE Manual states: Should a candidate exhibit any one behavior

in the “Failing Behaviors” column of the assessment sheet, he/she will be allowed to complete the Surgery section but will receive a failing grade

®

Fatal Flaws

CPE Manual States: Should the candidate commit a fatal flaw, the

examiner will interrupt the examination, correct the error to ensure safety of the animal and personnel, and award the candidate a failing grade.

The candidate will not be allowed to complete the Surgery section of the CPE.

®

Five Fatal Flaws

Fails to control significant hemorrhage Fails to create a secure closure of the

abdominal wall Unable to achieve and maintain an aseptic

field Causes significant damage to other

tissues/organs Exhibits any other behavior that puts the

animal’s life at risk

®

Radiology

Ongoing review by subject matter experts Development of consensus on types of

radiographs and radiographic lesions Asking experts: “What types of lesions

should a minimally competent entry-level veterinarian be able to identify and in what species?”

®

Goals of Radiology Review

Long-termDevelop a central bank of radiographs

Short-termUpdate competencies, skills/activities,

and observable behaviors for assessment of minimal entry-level competency in radiology

®

Additional Information

ECFVG Web sitewww.avma.org/education/ecfvg/default.asp

Contact ECFVG [email protected], ext 6675