Formative Assessment Case Studies (FACS) Shekhar Kumta, Jack Cheng, James Ware Chinese University of Hong Kong (Refer to CD for FACS engine and sample workshop case study) Hong Kong International Consortium
Dec 15, 2015
Formative Assessment Case Studies (FACS)
Shekhar Kumta, Jack Cheng, James WareChinese University of Hong Kong
(Refer to CD for FACS engine and sample workshop case study)
Hong Kong International Consortium
What Are FACS?
We have designed a user friendly engine for generating and delivering web-based, interactive, clinical scenarios
These case studies allow accommodation for multimedia features & require formative feedback to facilitate student learning.
RCTs have established the effectiveness of FACS
and students prefer using this learning and assessment model.
FACS engine provides a value added option for administering items from IDEAL’s non restricted (formative) bank.
Clinicians feel FACS provide a convenient means to generate & present a meaningful clinical context for one’s formative items.
FACS enables not only incorporation of, but enhanced use for formative items that are
Vignette-based A type
Vignette-based R- type
Key features based
FACS Strength: enhanced authenticity to the assessment
Evidence: one can assess student learning by testing only some elements that are essential to the resolution of that problem (i.e. the clinical problem’s key features).
Using key features is also an effective way of facilitating student learning (not just the assessment of their learning).
key features based assessments correlate better with problems solving ability and clinical reasoning than patient management problems
Option of using Key Features
How to Select Case Studies for FACS
• Identify areas where students frequently encounter difficulties understanding clinical concepts.
the clinical scenario needs to set a context so that application of knowledge becomes the focus for the virtual problem .
ensure steps within the problem enable strategic formative feedback in order to develop & facilitate understanding.
Workshop Objectives for FACS
Short Term Create a FACS
Write or incorporate existing formative items
Incorporate formative feedback
Intermediate
Install the FACS templates provided on your own or your institution’s computers / servers
Long Term Develop a bank of educationally relevant and validated FACS
for sharing among consortium members
+
Apply to several cases
1. Graduating Level
2. What do interns do?
3. Objectives = assessment
4. Practical issues
5. Consider theoretical issues
Generic Problem
Objectives
+
The Problem and Objectives
Generic problemGeneric problem Scenario Cases
Clinical detailAction List 1Feedback 1
Explanation 1 Action List 2 Feedback 2
etc
The Problem and Objectives
+
Chest pain
1. Clinical features
2. Prioritise management
3. Interpret investigations
4. Include discharge etc..
5. Anatomy of thorax
Generic Problem
Objectives
+
Suggest some notes
The Problem and Objectives
Generic problem ScenarioScenario
Writing the Scenario
INCLUDE:
1. Age and gender
2. Site of care
3. Summary of presentation
a) Keep generic at this stage
b) Onset, duration, severity
c) ~2 – 6 lines
4. Other (e.g., family history)
The Scenario:
• shall always include common info
• realism important
• summary form
Writing the Scenario
The Scenario:
Important, the same stem shall apply to several cases, e.g., pneumothorax and MI etc.
38 year old Chinese male with
a twenty year history of heavy
smoking is admitted to the
A&E Department with sudden
onset of chest pain one hour
before and is apparently
having difficulty in breathing.
Writing the Scenario
Choices of Cases
Generic problem Scenario CasesCases
+
Case 1Create several related cases:
• common scenario
• common cases
• common objectives
+
Case 2
Case 3
+
Case 4 . . .
Choices of Cases
+
Pneumothorax, tensionThe Cases
+
Pneumothorax, simple
Myocardial infarction
+
Acute oesophagitis
Choices of Cases
Scenario Cases Clinical detailClinical detail
Writing Clinical Detail
Case 1: Pneumothorax
Consider site
Usually vital signs
Important info:
a) Specific clinical findings
b) + special tests
Clinical detail
Clinical images
+
Writing Clinical Detail
Clinical detail
With Added value
HR: 120bpmBP: 80/60 mm HgRR: 30 / minT: 37.5o C
Hyper-resonant Rt., absent BS and reduced respiratory movement Rt.,
Writing Clinical Detail
Writing an Action List
Scenario Cases Clinical detail
Action List 1Action List 1
+
Lead-in
1. Brief
2. Likely choices
3. Pick one best response
4. Ensure four or more
Include
Up to “n” possible actions
Writing an Action List
+
What is your next action?Lead-in
Incorrect
Correct choice
Incorrect
+
1.Order chest radiograph
2. Insert a chest drain
+
3. Order a chest CT
Writing an Action List
Scenario Cases Clinical detail
Action List 1Feedback 1 for Feedback 1 for incorrect choicesincorrect choices
Writing Feedback
+
For all incorrect choices
1. Why inappropriate
or
2. Trigger further thought
3. Linkages to resources
4. Another image?
Feedback
Included
Writing Feedback
+
1. Life threatening situation . . .For each feedback that is incorrect give a summarised form of info.
Will offer re-entry assessment after incorrect choices
+
2. Quite incorrect . . .
3. No place for definitive . . .
+
4. Tests unnecessary . . .
Writing Feedback
Scenario Cases Clinical detail
Action List 1Feedback 1 for incorrect choices
Explanation 1 for Explanation 1 for correct choicecorrect choice
Explanation for Correct Choice
REPEATED SEQUENCE
+
Summarised
1. Video of procedure
2. Text for action
or
3. Result (s) of investigation
Explanation
Added value
Explanation for Correct Choice
Scenario Cases Clinical detail
Action List 1Feedback 1 for incorrect choices
Explanation 1 for correct choice
Action List 2Action List 2
Writing an Action List
+
Lead-in
1. Brief
2. Likely choices
3. Pick one best response
4. Ensure four or more
Action List
Up to “n” possible actions
Writing an Action List
+
What is your next action?Lead-in
Partially incorrect
Correct choice
Incorrect
+
1.Order chest radiograph
2. Admit for observation
+
3. Discharge after . . .
Writing an Action List
Scenario Cases Clinical detail
Action List 1Feedback 1 for incorrect choices
Writing Feedback
Explanation 1 for correct choice
Action List 2 Feedback 2 for Feedback 2 for incorrect choicesincorrect choices
+
For all incorrect choices
1. Why inappropriate
or
2. Trigger further thought
3. Linkages to resources
4. Another image?
Feedback
Included
Writing Feedback
+
1. Not an immediate priority . . Don’t forget the further assessment to re-enter at the last action point
+
2. Pt. needs monitoring . . .
3. Too early for discharge . . .
+
4. Review later for need . . .
Writing Feedback
Scenario Cases Clinical detail
Action List 1Feedback 1 for incorrect choices
Explanation 1 for correct choice
Action List 2 Explanation 2 for Explanation 2 for correct choice (s)correct choice (s)
Feedback 2 for incorrect choices
Explanation for Correct Choice
+
Summarised
1. Video of procedure
2. Text for action
or
3. Result (s) of investigation
Conclusion
Added value
Explanation for Correct Choice
+
Use relevant clinical info
1. MCQ format
2. + vignette
3. R type
4. Key features etc . . .
Stem
Re-entry Assessment
Options
+
Scenario Cases Clinical detail
Explanation 1 for correct choice
Action List 2
Explanation for Correct Choice
correct choice
# failed tries decides if exit
Feedback 2 for incorrect choices
incorrect choices
Action List 1
Assessment
CREATING A FACS
DEMONSTRATION