IDEA What it is and How to Implement the System Texas A & M, February 2013 Shelley A. Chapman, PhD Senior Educational Consultant
Feb 21, 2016
IDEAWhat it is and How to Implement the SystemTexas A & M, February 2013
Shelley A. Chapman, PhDSenior Educational Consultant
IndividualDevelopmentEducationalAssessment
Teaching ImprovementFaculty Evaluation
Curriculum ReviewProgram AssessmentAccreditation
What is teaching effectiveness?
Being OrganizedBeing PromptBeing Clear
Relating course material
to real life situationsRequiring
critical thinking
Forming learning teams
Introducing
stimulating ideas
Inspiring students to set and achieve goals
Most Surveys
How well do the instructor’s methods resemble those of a “model” teacher?
How well do students rate their progress on the types of learning the instructor targeted?
Teaching Effectiveness
What is teaching effectiveness?
Philosophy of IDEA
Primary indicant ofTeaching Effectiveness
= Facilitating Learning
Conditions for Good Use
The instrument • Targets learning • Provides suggested action steps for teaching improvement• Has evidence for validity
Conditions for Good Use
The Faculty• Trust the process• Value student feedback• Are motivated to make
improvements
Conditions for Good Use
Campus Culture• Teaching excellence - high
priority• Resources to improve -
provided• Student ratings - appropriate
weight
Conditions for Good Use
The Evaluation Process• 30-50% of evaluation of teaching• 6-8 classes, more if small (<10)• Not over-interpreted
(3-5 performance categories)
Reflective Practice using Individual Reports
Collect Feedback
Interpret Results
Read & Learn
Reflect & Discuss
Improve
IDEA resources that are keyed to reports
Talk with colleagues
Try new ideasOnline, Paper
What the reports sayand what they mean
Underlying Philosophy of IDEATeaching effectiveness is determined primarily by students’ progress on the types of learning the instructor targets.
Faculty Information Form
Student Learning ModelSpecific teaching behaviors are associated with certain types of student progress under certain circumstances.
Student LearningTeaching Behaviors
Circumstances
Student Learning Model: Diagnostic Form
Student LearningItems 21-32
Teaching Behaviors
Items 1-20
CircumstancesStudents: Items 36-39, 43
Course: Items 33-35Summary Items: 40-42Research Items: 44-47Up to 20 extra items
Student Learning Model: Short Form
Summary Measures: Items 16-18Experimental Questions: Items 1420 Additional Questions
Student LearningItems 1-12
Teaching Behaviors
CircumstancesStudents: Items 13-15
FIF: Selecting Objectives• 3-5 as “Essential” or “Important”
• Is it a significant part of the course?• Do you do something specific to
help students accomplish the objective?
• Does the student’s progress on the objective influence his or her grade?
Be true to your course.
Faculty Information Form
Common Misconception #1
Students are expected to make significant progress on all 12 learning objectives in a given course.
Common Misconception #2
Effective instructors need to successfully employ all 20 teaching methods in a given course.
Relationship of Learning
Objectives to Teaching Methods
Common Misconception #3
The 20 teaching methods items should be used to make an overall judgment about teaching effectiveness.
Faculty Evaluation
Faculty Information Form: Discipline Codes
www.theideacenter.org/DisciplineCodes
Faculty Information Form: Local Code
Course Description Items (FIF)• Used for research• Best answered toward end of term• Do NOT influence your results
Bottom of Page 1
Top of page 2
IDEA Online
FIF Online Delivery
• Reminders are delivered by email• Start/end dates are determined by Institution• Access is unlimited while available• Questions can be added • Objectives can be copied
Copying Objectives
Student Survey Online Delivery
• Link is on Howdy• Reminders are sent by email• Start/end dates determined by Institution• Submission is confidential and restricted to one
Online Response Rates – Best Practices
• Create value for feedback• Prepare Students• Monitor and
Communicate
Example: Course Syllabus
Objective 3: Learning to apply course material (to improve thinking, problem solving, and decisions)
Students will be able to apply the methods, processes, and principles of earth science to understanding natural phenomena
Students will think more critically about the earth and environment
Objective 8: Developing skill in expressing myself orally or in writing
Students will be able to present scientific results in written and oral forms
IDEA Center Learning Objective
Course Learning Outcomes
Diagnostic Report Overview1. How did students rate their
learning experience?
2. What contextual factors impacted those ratings?
3. How do my scores compare to: IDEA, discipline, and institution?
4. What might I do to facilitate better learning for my students next time?
Your Average (5-point Scale)
Raw Adj.
A. Progress on Relevant Objectives1
Four objectives were selected as relevant (Important or Essential—see page 2)
4.1 4.3
1If you are comparing Progress on Relevant Objectives from one instructor to another, use the converted average.
1. How did Students Rate their Learning?
ProgressOnRelevantObjectives
4
4.3 + 4.34.14.23.6
5
Summary Evaluation: Five-Point Scale Report Page 1
Your Average Score
(5-point scale)
Raw Adj.
A. Progress on Relevant ObjectivesFour objectives were selected as relevant (Important or Essential—see page 2)
4.1 4.3
Overall Ratings B. Excellent Teacher 4.7 4.9
C. Excellent Course 4.1 4.4
D. Average of B & C 4.4 4.7
Summary Evaluation(Average of A & D) 4.3 4.5
50%
25%25%
2. What contextual factors impacted those scores?
3. How do my scores compare to: IDEA, Discipline, Institution?
Comparisons (Norms): Converted Averages
4. What might I do to facilitate better learning next time?
Page 2: What did students learn?
Page 3: Suggested Action Steps
#16#18#19
POD-IDEA Notes onIDEA Website
POD-IDEA Notes
• Background• Helpful Hints• Application for online
learning• Assessment Issues• References and
Resources
IDEA Papers
Resources for • Faculty Evaluation• Faculty Development
IDEA Terminology• Student Ratings of Instruction = student survey• FIF = Faculty Information Form• OCC = On Campus Coordinator• Sub-OCC = Person who works with the OCC on campus• GSR = Group Summary Report• Aggregate Data File = Excel spreadsheets of all data• Benchmarking Report • Discipline Code = modified CIP codes • Local Code = code for creating groups• Converted Averages = T Scores• Adjusted Scores = Scores that take into consideration
variables outside the control of the instructor