Just-in-Time Teaching: Tilting Classes Across the Academy Jeff Loats, Arlene Sgoutas, & Randi Smith (Physics) (Women’s Studies) (Psychology) Colorado Learning and Teaching with Technology August, 2016
Just-in-Time Teaching:Tilting Classes Across the
AcademyJeff Loats, Arlene Sgoutas, & Randi Smith
(Physics) (Women’s Studies) (Psychology)
Colorado Learning and Teaching with Technology
August, 2016
Presentation Overview• What is it?
Introduction to Just-in-Time Teaching (JiTT)
• Does it work? Data from our courses
• How do I get started? Recommendations for implementing JiTT
In your teaching do you have a method for holding students accountable for preparing for class?
A) I don’t, but I ask/threaten really well.B) I use a paper method (quiz, journal, others?)C) I use a digital method (clickers, others?)D) I have some other method.
3
18%51%15%17%
others)
Consider a typical day in your class. What fraction of students did their preparatory work before coming to class?
A) 0% - 20%B) 20% - 40%C) 40% - 60%D) 60% - 80%E) 80% - 100%
4
29%34%29%13%5%
( others)
Student Preparation Research
Quotes from Sappington, Kinsey, & Munsayac (2002)
"72% of Connor-Greene’s (2000) sample reported that they rarely or never read their assignments by the due date.”"Burchfield and Sappington (2000): On any given day, less than a third of students in this population had adequately prepared for class."Recent USPIRG survey: 70% of students admit that they sometimes don’t evenobtain required textbooks.
JUST-IN-TIME TEACHINGOnline pre-class assignments called WarmUpsFirst half - Students• Conceptual questions, answered in sentences• Graded on thoughtful effortSecond half - Instructor• Responses are read “just in time”• Instructor modifies that day’s plan accordingly.• Aggregate and individual (anonymous) responses
are displayed in class.
Learner Teacher
JUST-IN-TIME TEACHINGA different student role:• Actively prepare for class
(not just reading/watching)• Actively engage in class• Compare your progress & plan accordinglyA different instructor role:• Actively prepare for class with these humans
(not just going over last year’s notes )• Modify class accordingly• Create interactive engagement opportunities
Learner Teacher
TOUTED BENEFITS OF JITT• Student and instructor preparation• Supports development of metacognitive skills• Brings student voices into the classroom• Promotes engagement with “higher-level”
questions (cf. Bloom’s taxonomy)• Consistent with other research-based
instructional strategies (learner-centered, universal design, etc.)
Example Questions & Responses
Suppose you are interested in researching why and how some women fake orgasms. How would you design a study to answer those questions?
Perhaps I’d tell a random group of women to fake an orgasm. Additionally, a control group that is not told to fake an orgasm should be included. After both groups’ ensuing sexual encounters, ask very specific and pointed questions about sexual arousal, climax, or the lack thereof, and the experience overall.
Identify and define the three different conceptualizations of gender and sex that have characterized the study of gender in recent years.
1.) Gender as equivalent to sex: gender is "rooted in biological characteristics," or each animal is assigned a gender according to their sex; females are feminine, males are masculine. 2.)Gender as separate from sex: sex is the physical characteristics (reproductive roles), and gender is "personality roles" constructed from socialization. 3.) Gender as the assignment of meaning to bodies: That gender and sex are both social constructs.
Sounds good in theory, but does it actually work?
Convince me it’s worth my time…
Our Data: Set• We examined data from over 20 courses across
5 disciplines, over 500 students total– Upper- and lower-division courses– Larger and smaller courses– General studies, required major courses,
elective courses• Our implementations of JiTT vary
– Weekly vs. biweekly– 5-20% of final course grade
Our Data: Participation Rates• Response rates (averaged by class) vary from
~50% to ~80% across our two-dozen courses.
• Recall: The average preparation rate reported by Burchfield & Sappington (2000) was ~33%.
Week 1
2 3 4 5 6 7 8 9 10 11 12 13 14 150%
20%
40%
60%
80%
100%
WMS 390Y (N = 19)Avg. = 77%
Week 1
2 3 4 5 6 7 8 9 10 11 12 13 14 150%
20%
40%
60%
80%
100%
PSY 2850 (N = 23)Avg. = 68%
~Daily WarmUps
~Weekly WarmUps
Day 2
4 6 8 10 12 14 16 18 20 22 24 26 280%
20%
40%
60%
80%
100%
Physics 2711 (N = 24)Avg. = 72%
Day 2
4 6 8 10 12 14 16 18 20 22 24 26 280%
20%
40%
60%
80%
100%
Physics 1000 (N = 22)Avg. = 63%
Our Data: WarmUp Correlations• Bulk correlation of Total Grade ↔ WarmUps
– ranges from to – Combined* correlation: (N = 581)
• Hattie (2009) noted that effect sizes for most pedagogical techniques are small to medium (see, e.g.,). From Cohen (1988): is medium effect is large effect
Our Data: Correlation Spread
*Correlations were converted to Fisher Zs, averaged and converted back to Pearson r.
WMS390z
WMS390y
WMS1001
PSY3050
PSY2850
PSY2850
PSY2410
PSY2410
PSY1001
PSY1001
PSY1001
PSY1001
PHY4820
PHY3620
PHY3620
PHY3011
PHY2020
phy1000
phy1000
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
Total Grade vs. WarmUp Correlations by Class
r = 0.57Combined* correlation
Our Data: Testing Personality Effects• In trying to tease out causality, we are looking
to control for personality traits. – The Big Five: openness, conscientiousness,
extraversion, agreeableness, and neuroticism.– For example: Does the WarmUp ↔ Total
Grade correlation just come from conscientiousness?
• So we cajoled 77 students into an extensive personality assessment.
Our Data: Personality Effect ResultsFrom our 77 students we have
WarmUp ↔ Total Grade: (puzzling)
Variable Statistical Significance
Extraversion 0.460Agreeableness 0.258Conscientiousness 0.227Neuroticism 0.483Openness 0.034WarmUp Grade 0.032
PRELIMINARY RESEARCH SUMMARY Participation rates appear to be quite stable
across disciplines, levels, course size, etc. The importance of WarmUps to learning is not
explained by student conscientiousness.
Is JiTT especially effective in scaffolded topics?
Are students that completed the personality assessment a skewed sample? Why is their WarmUp ↔ Total Grade correlation so low?
Wow—you guys are so convincing. I’m totally sold on this JiTT stuff. Now what?First… pause for questions?
A POSSIBLE PLANChoose one course you will teach next term.A. Write two questions for each class meeting:
1. One lower-level (maybe multi-choice?).One higher-level (sentences).
2. Give yourself 10 minutes to write each oneB. Write a standard (1st) metacognitive question
(What was most interesting or confusing …?)C. Discuss one question at the top of class, and one
in the middle. Use the metacognitive responses as break points or highlights.
OUR SUMMARYJiTT may be among the easiest research-based instructional strategies that you can consistently integrate into your teaching.
JiTT is a research-based instructional strategy that addresses often-neglected areas.
JiTT brings other benefits to students and instructors beyond those measured in most studies.
Strongly Disagree
Strongly Agree
1 2 3 4 5Completing the warm-ups in this class helps to prepare me for class meetings.I do more of my assigned readingfor this class than I would if thereweren't warm-ups assigned.The warm-ups are worth the timeand effort required to completethem because they help me learn.I find it easier to learn the materialin this class because of the warmups.The warm-ups feel like a waste ofmy time.
2.19
3.63
3.89
3.58
3.95
STUDENT SURVEY RESPONSES I
Strongly Disagree
Strongly Agree
1 2 3 4 5The warm-ups are worth enoughpoints/credit toward my final gradein this class, given the time andeffort they require.Overall, I find I have to spend lesstime studying for exams because ofthe learning I've done for the warmups.
I'm better able to determine what Iunderstand well and what I need tostudy more because of the warmups.
My instructor uses the warm-ups ina meaningful way to tailor classmeetings based on our needs.
The warm-ups are a nuisance tocomplete. 2.58
4.09
3.28
2.74
3.59
STUDENT SURVEY RESPONSES II
YOUR SUMMARYIf you want to implement JiTT, what is your next concrete action?
Jeff Loats: [email protected], @JeffLoatsArlene Sgoutas: [email protected] Randi Smith: [email protected]: www.slideshare.net/JeffLoats
Thanks for your attention!
JITT REFERENCES & RESOURCESSecond Book (interdisciplinary):Simkins, Scott and Maier, Mark (Eds.) (2010) Just in Time Teaching: Across the Disciplines, Across the Academy, Stylus Publishing.Original Book (physics examples):Gregor M. Novak, Andrew Gavrin, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time Teaching: Blending Active Learning with Web Technology. Prentice Hall. Upper Saddle River NJ.K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to Actually Read the Assignment. Teaching Sociology, Vol. 32 (No. 4 ). pp. 385-390. Published by: American Sociological AssociationStable URL: http://www.jstor.org/stable/3649666S. Linneman, T. Plake (2006). Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol. 54 (No. 1)Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18Sappington J, Kinsey K and Munsayac K (2002) Two studies of reading compliance among college students. Teaching of Psychology 29(4): 272–274.http://orgs.bloomu.edu/tale/documents/reading_sappington_twostudies.pdfLouis Deslauriers, Ellen Schelew and Carl Wieman (2011). Improved Learning in a Large-Enrollment Physics Class. Science, Vol. 332 no. 6031 pp. 862-864 DOI: 10.1126/science.1201783https://www.sciencemag.org/content/332/6031/862.abstractFreeman S, et al. (2014) Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci USA 111:8410–8415.http://www.pnas.org/content/111/23/8410.abstract
ON-DEMAND SLIDES