Triangulation in Outcome-Based Education (OBE) Evaluation Ma. Cecilia D. Alinea, MD, MHPEd, FPPS Clinical Associate Professor, Department of Pediatrics College of Medicine- Philippine General Hospital University of the Philippines Manila Undergraduate Pediatric Education Curriculum (UPEC) 4th Q&A Forum PPS Annual Convention
34
Embed
Triangulation in OBE Evaluation FINAL · Triangulation in Outcome-Based Education (OBE) Evaluation Ma. Cecilia D. Alinea, MD, MHPEd, FPPS Clinical Associate Professor, Department
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Triangulation in Outcome-Based Education
(OBE) Evaluation
Ma. Cecilia D. Alinea, MD, MHPEd, FPPS Clinical Associate Professor, Department of Pediatrics
College of Medicine- Philippine General Hospital University of the Philippines Manila
Undergraduate Pediatric Education Curriculum (UPEC)4th Q&A Forum
PPS Annual Convention
Road Map• Roles of assessment
• Assessment in relation to OBE curriculum
• Learning targets
• Triangulation as a process
• Different assessment methods and tools
• Application to PPS OBE curriculum
How can I do better on the
next test?
How can I gauge whether my class is understanding this
lesson?
Classroom:
• Support learning during
instruction • Certify
learning after instruction
Examples:
• Descriptive feedback
• Work products• Unit quizzes
and exams
School and Region:
• Evaluate program
effectiveness • Measure student
growth • Support
placement, intervention, remediation
Examples:
• Diagnostic tests• Benchmark assessments
• Interim growth measures
National:
• Understand achievement
trends • Evaluate
school performance • Determine
access & equality
Examples:
• Statewide accountability
tests • National
Assessments • International
Assessments test
How can I tell if my 3rd year med students
are all learning?
How can I identify which schools need
the most support w/ the OBE curriculum?
How do we raise the standard in our medical
schools all over the country?
How do we ensure that all med students have equal access to quality education?
Assessment Literacy
Knowledge and Skills of students UNCLEAR
How can you measure with quality by an assessment?
Teachers and Students UNCLEAR on what to learn
How can an assessment provide guidance on meeting learning
objectives?
LEARNING TARGETS
TeachersStudents
Outcome Based Education
Shumway J and Harden R. AMEE Guide 25: The Assessment of learning outcomes for the competent and reflective physician, Medical Teacher, Vol. 25, No. 6, 2003, pp. 569–584
Three Circle Model representing educational outcomes
Performance of task
Approach to task
Professionalism
Competent and Reflective Physician
Learning Targets• Content standards,
year level expectations, lesson objectives
• Communicated clearly, and in advance to students
Clear areas to beassessed
Tasks/ activities reflect targets
Assessment methods that fit targets
CLARIFYING LEARNING TARGETS
Triangulation: Process
Informed educational
decision making: ASSESSMENT
RESULTS’ INTERPRETATION
Creates a MORE COMPLETE
PICTUREof student learning /
needs
All results with SIMILAR
CONCLUSIONS—> More confident in assessment and how to address student’s needs
• Begin with your institutional story- mission, strategic plan (institutional priorities), program assessment in place (Nelson, 2010)
• Identify sources of assessment- objectives for tool, who does the assessment, collection, formative vs summative
• Triangulate- quantitative vs qualitative, competencies assessed
• Decision-making- formative vs summative, status of student, needs, how to meet needs
Case and Swanson (1993)Extended Matching Items● Description - consist of theme description, a series of options (up to
26), a lead-in and a series of short cases or vignettes
● Indications - assessment of decision making or problem solving skills - cover large knowledge base/hour of testing time - large number of students are to be tested
Case and Swanson (!993)Extended Matching ItemsTheme: Diagnosis (Respiratory Tract) Options: a. Asthma d. Laryngotracheomalacia g. Cystic fibrosis b. Pneumonia e. Epiglottitis h. Diphtheria c. Tuberculosis f. Bronchiolitis i. Pharyngitis
Lead-in: For each of the ff cases select the most likely
diagnosis
Case and Swanson (1993)Extended Matching ItemsVignettes: 1. A 4 year old boy presents in the ER with cyanosis. He has a 1 week
history of cough and fever, associated with poor appetite. On PE, RR= 85/min, with intercostal and subcostal retractions, rales on all lung fields. On x-ray you note infiltrates in the lung parenchyma.
2. A 2 year old boy was brought to the OPD for weight loss. He has on & off cough for the past 3 months with poor appetite. His lolo, the primary caregiver, has been taking “ 4 vitamins for the lungs” for the past 2 months w/ poor compliance.
3. A 2 month old baby was brought to the ER due to cough, nasal congestion and fever of 3 days duration with an occasional “whistling sound.” He had rapid breathing and mild chest indrawing. On PE you noted intercostal retractions and wheezing.
a. Asthma d. Laryngotracheomalacia g. Cystic fibrosis b. Pneumonia e. Epiglottitis h. Diphtheria c. Tuberculosis f. Bronchiolitis i. Pharyngitis
a. Asthma d. Laryngotracheomalacia g. Cystic fibrosis b. Pneumonia e. Epiglottitis h. Diphtheria c. Tuberculosis f. Bronchiolitis i. Pharyngitis
Written Assessments
Wilson RB, Case, SM. Extended MatchingQuestions: An Alternative to MCQs or Free Response Questions. Journal of Veterinary Medicine Education. 1993: Vol. 20, Number 3
Constructed Response Questions (CRQ)
• Tests contain questions that require the test taker to produce the answer
• Take longer to answer than MCQs, limited number of questions to answer at a given time
• Performance may vary from one set of questions to another
• Responses tend to be time-consuming to score
• Higher cost of testing and longer time required to compute and report the scores
CRQ Scoring• Requires judgment• Different scorers can award different scores
to same response
ANALYTICAL SCORING
Rubric contains features of response to focus on and points to award to each
feature
HOLISTIC SCORING
Rubric contains statements describing the
characteristics of a typical response at each score level
Bordage (1997)Key Features ExamA 5 year old girl was brought to you for high grade fever of 5 days
duration. The mother noted she had poor appetite and activity and would complain of abdominal pain and vomiting. She noted that her stools were “blackish” in color. On PE, BP was 80/50, HR 128, RR 35, with note of flushed skin. No overt bleeding noted. Abdominal exam showed generalized tenderness on palpation.
List down 3 differential diagnoses for this case: 1. Dengue Fever/ Hemorrhagic Fever (1 pt) 2. Typhoid Fever (1 pt) 3. Systemic Viral Illness (1) pt What are the first 3 management steps that should be performed? 1. Insert IV fluids for hydration (1 pt) 2. Monitor bleeding parameters through CBC (1 pt) 3. Insert an NGT (0.5 pt)
Clinical/ Practical Assessments
• Long cases • Practical exam • Spot exam • Objective structured clinical examination (OSCE) • Objective structured practical examination (OSPE) • Objective structured long examination record
(OSLER) • Group objective structured clinical examination