“DESIRABLE DIFFICULTIES” STUDYING LEARNING REMEMBERING J Baucom, Landmark College, Summer Institute 2017
“DESIRABLE DIFFICULTIES”
STUDYING LEARNING
REMEMBERING
J Baucom, Landmark College, Summer Institute 2017
Zeus, Mnemosyne, and the Nine Muses(Anton Mengs, 1761)
Six Highly Effective Study PracticesSpace Your Studying
Vary the Conditions of Study
Test Yourself
Write Questions
Draw Visuals
Summarize What You Learn
Agenda:
u Important features of memory
u Why difficulties in learning are good
u 3 study strategies that build on how memory works
u Selected research studies
u What insights into studying tell us about teaching
u Educational implications, questions
QUIZu 1. What was the name of your 6th grade math teacher?
u 2. Who was the 25th President of the United States?
u 3. What is the name of the person you introduced yourself to at the beginning of this session?
Hermann Ebbinghaus (1885)u First to apply experimental procedures,
mathematical models, and statistics to the study of learning and memory
u Conducted a series of rigorously controlled experiments between 1879-1884 with a single subject
Ebbinghaus: 4 Major Contributions
Forgetting Curve
• What is lost
Relearning Effect
• What is saved
Spacing Effect
• How it’s distributed
Overlearning
• Surplus practice
0%
20%
40%
60%
80%
100%
120%
0 19 minutes 63 minutes 8 hours 24 hours 2 days 6 days 31 days
PERC
ENTA
GE
OF
INFO
RMA
TION
RE
LEA
RNED
INTERVAL BETWEEN ORIGINAL LEARNING AND MEMORY TEST
Ebbinghaus's Forgetting/Retention Curve
A New Theory of Disuse Robert and Elizabeth Bjork (1992)
DisuseDecay
RetrievalStrength
Two Distinct Memory StrengthsStorage: How well learned
Strong Weak
Retrieval: How accessible
Strong Weak
High SS/Low RS
q childhood street address
q Distributive law in mathematics
Low SS/ High RS
q current hotel room number
q Ebbinghaus: what percentage of memory loss occurs in first 20 minutes after learning?
Low SS/Low RS
q old hotel room number
q 25th US president
High SS/High RS
q current street address
q how to calculate a percentage
Key Design Features of Storage
Unlimited Doesn’t Diminish
Expands with Use Latent
Value of Storage Strength
Ø Enhances gain of RS
Ø Slows the loss of RS
Ø Enables us to recognize or relearn info
What Strengthens Storage?
SS
RepeatedPractice
Meaningfulness
Connection to Existing
Knowledge
Key Design Features of Retrieval
Limited, FrailDetermines
Current Performance
Strengthens with Difficulty
Changes Memory
Determinants of Retrieval
RS
AbsoluteStrength
RelativeStrength
ContextCues
u Who was the 25th President of the United States?
u Which President was assassinated, resulting in Theodore Roosevelt becoming President?
u William McKinley
Adaptive Value of Disuse Theory
Takeaways/Educational Implications
Forgetting is good
How do we boost RS?
High RSLow SS
Beware “illusion of mastery”
Good teaching=
SS +++
What Is A Desirable Difficulty?
u “training conditions that are difficult and appear to impede performance during training but that yield greater long-term benefits than their easier training counterparts” (Bjork, 2016)
u “…conditions that produce the fewest errors during learning (like massed practice…) can lead to very poor long-term retention” (Karpicke and Roediger 2008)
uStudy harder?
The Cognitive Reflection Test (Frederick 2005)
u 1. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.
How much does the ball cost?
u Answer:
u Alter et al (2007) administered CRT to Princeton studentsu Avg Score: 1.9/3
u With altered font? Avg score: 2.45 out of 3
Strategy/Desirable Difficulty 1: Space Your Studying (Distributed Practice)
u Spread out practice of material over time rather than in one large time block
u Opposite of cramming
Spaced Study or Distributed Practice
Mon Tues Wed Thurs Total StudyTime
Fri-Test Day
John 4 hours
Susan 4 hours
Fernando 4 hours
Spaced Study or Distributed Practice
Mon Tues Wed Thurs Total StudyTime
Fri-Test Day
John 4 hours 4 hours
Susan 2 hours 2 hours 4 hours
Fernando 1 hour 1 hour 1 hour 1 hour 4 hours
Spaced Study or Distributed Practice
Mon Tues Wed Thurs Total StudyTime
Fri-Test Day
John 4 hours 4 hours C-
Susan 2 hours 2 hours 4 hours B-
Fernando 1 hour 1 hour 1 hour 1 hour 4 hours A
Cepeda et al. 2006 839 assessments of spaced
study
317 different experiments
14,000 participants
average benefit of 15% on test performance for
spaced study over massed study
u Spacing between study sessions should be 10-20% of the desired retention interval (the delay between final study session and criterion task) Dunlosky et al., 2013
u To remember info 30 days later, aim for 3-6 days between study sessions
Moulton, Resnick et al (2006) “Teaching Surgical Skills: What Kind of Practice Makes Perfect?”
u University of Toronto Department of Surgeryu Taught complex surgical skill using different schedules of instruction:
Massed Training: 4 sessions in one day
Distributed Training: 1 session per week, over 4 weeks
Immediately After Training
•Both groups equal performance
One Month Later
•DT group better on retention and transfer
Why Does Spacing Work?
RelearingEffect
Forgetting Occurs
Desirable Difficulty
Ebbinghaus on Spacing Effect
u …the method naturally employed in practice agrees. The schoolboy doesn’t force himself to learn his vocabularies and rules altogether at night, but knows that he must impress them again in the morning. A teacher distributes his class lesson not indifferently over the period at his disposal but reserves in advance a part of it for one or more reviews” (p. 89)
Implications Beyond School and College
Educational Implications/Reflection
uWriting Exercise: What are the implications for spacing in the design and implementation of our educational models?
uWriting Exercise: Knowing what you know about spacing effects, how might you apply this powerful strategy in your work?
Strategy/Desirable Difficulty 2:
uVary the Conditions of Your Learning
uPhysical Context
uCognitive Context (questions, visuals, summaries)
u Informational Context: Interleaving
Why Create Variation?
u Builds a richer network of connections and associations, “memory landmarks”
u Reduces context dependency, encoding specificity (Tulving, 1973)
Physical Context: Studying and Testing (Smith, Glenberg &Bjork (1978)
Study Session 1 Study Session 2 Testing
Group 1 Context A Context A Context C(15.9 words, 39%)
Group 2 Context A Context B(completelydifferent)
Context C(24.4 words, 61%)
Task: Study a list of 40 words
Adding Variation: Interleavingu Alternate skills or concepts within a given time period, rather than studying
each one in a large blocku
u A-A-A, B-B-B, C-C-C
A-B-C, A-B-C, A-B-C or A-B-C, B-A-C, C-B-A
u Applies to motor skills, semantic knowledge, procedural knowledge, arts, music, athletics
u Kang and Pashler (2012), college students studied 40 paintings by three artists (Blencoe, Lindenberg, O’Shea) and their styles in interleaved, massed block, or spaced block conditions
u Interleaved group best able to recognize painters’ styles on a transfer test
Interleaving Math Instruction (Rohrer et al, 2015)
u Over a 3 month period, 126 7th graders were taught how to graph linear equations and how to find the slope of a line
u Some classes = homework with interleaved graph, blocked slope problems
u Other classes = homework with interleaved graph and blocked slope problems
0
10
20
30
40
50
60
70
80
90
I day delay 30 day delay
Results of Rohrer Experiment 2015
Interleaved Blocked
Why Does Interleaving Work?
Educational Implications/Reflectionu How well do we “mix up” and create varying conditions in our
teaching? Are we too cautious about mixing it up?
u Do we interleave instruction or teach in blocked units?
u Interleaving is how the “real world” works!
u Writing Exercise: Knowing that varying the conditions of learning can impact retention, how might you apply this desirable difficulty in your work?
Strategy/Desirable Difficulty 3: Test Yourself
u Test-enhanced learning
u “Testing effect”
u One of the best study practices (Dunlosky et al. 2013)
Studying vs Testing
0102030405060708090
5 Minutes 1 Week
Prop
ortio
n of
Idea
s Rec
alle
dd
Retention Interval
Roediger & Karpicke 2006
SSSS SSST STTT
Why does testing improve learning?
Demands Retrieval
ModifiesMemory
Increases SS and RS
How Can Students Test Themselves?
•Summaries•Visuals
•SMART Cards•Quizlet.com
•Reciprocal Teaching•Practice Tests
Educational Implications/Reflection
u Is the testing effect utilized as a teaching tool often enough?
u Perception: 1) purpose, value of testing 2) low-stakes vs high stakes testing
u Writing Exercise: Knowing what you know about the testing effect, how might you apply this powerful strategy in your work?
Putting It All Together: A Study Plan
MonMar 11
Mon Mar 18
MonMar 25
MonApr 1
Sun Apr 7
MonApr 8Test Day
Karen 1 hour 1 hour 1 hour 1 hour 1 hour A+++
Where? Context AEAB108
Context BLibrary
Context AEAB108
Context CDining Hall
Context AEAB108
Context AEAB108
How? Study
Interleave
Self Test
Interleave
Self Test
Interleave
Self Test
Interleave
Study, night before test,then sleep
Real Test
Spaced Study
u Writing Exercise: What are the implications for spacing in the design and implementation of our educational models?
u Writing Exercise: Knowing what you know about spacing effects, how might you apply this powerful strategy in your work?
Interleaving
u Writing Exercise: Knowing that varying the conditions of learning can impact retention, how might you apply this desirable difficulty in your work?
Testing Effect
u Writing Exercise: Knowing what you know about the testing effect, how might you apply this powerful strategy in your work?
Sources Cited
Alter, A.L., Oppenheimer, D.M., Epley, N., Eyre, R.N. (2007). Overcoming intuition: metacognitive difficulty activates analytic reasoning. Journal of experimental psychology: general, 136, (4), 569-576. Bjork, R.A. & Bjork, E.L. (1992). A new theory of disuse and an old theory stimulus fluctuation. In A. Healy, S. Kosslyn, & R. Shiffrin (Eds.) From Learning Processes to Cognitive Processes: Essays in Honor of William K. Estes, (Vol. 2, pp. 35-67). Hillsdale, NJ: Erlbaum. Bjork, R.A. & Bjork, E.L. (2016). UCLA: Bjork learning & forgetting lab. bjorklab.psych.ucla.edu/research.html Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of experimental psychology: Learning, memory and cognition, 36 (5), 1118-1133. Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354-380. Cepeda, N.J., Coburn, N., Rohrer, D., Wixted, J.T., Mozer, M.C., & Pashler, H. (2009). Optimizing distributed practice: Theoretical analysis and practical implications. Experimental psychology, 56 (4), 236-246. Dempster, F. N. (1988). The spacing effect: A case study in the failure to apply the results of psychological research. American psychologist, 43 (8), 627-635. Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J., & Willingham, D.T. (2013). Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychological science in the public interest, 14 (1), 4-58. Ebbinghaus, Hermann (1964). Memory: A contribution to experimental psychology. (Henry A. Ruger & Clara E. Bussenius, translators.). New York: Dover Publications, Inc. (Original work published 1885) Frederick, S. (2015). Cognitive reflection and decision making. Journal of economic perspectives, 19 (4), 25-42. Grant, H. M., Bredahl, L. C., Clay, J., Ferrie, J., Groves, J. E., McDorman, T. A., & Dark, V. J. (1998). Context-dependent memory for meaningful material: Information for students. Applied cognitive psychology, 12, 617-623. Kang, S. H.K., & Pashler, H. (2012). Learning painting styles: spacing is advantageous when it promotes discriminative contrast. Applied cognitive psychology, 26 (97-103).
Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. Science, 319, 966-968. doi:10.1126/science.1152408 Kerr, R. & Booth, F. (1978). Specific and varied practice of motor skill. Perceptual and motor skills, 46 (2) 395-401. Moulton, C-A.E., Dubrowski, A., MacRae, H., Graham, B, Grober, E. & Reznick, R. (2006). Teaching surgical skills: what kind of practice makes perfect? Annals of surgery, 244 (3), 400-409. Nantais, K.M., & Schellenberg, G.E. (1999). The Mozart effect. Psychological science, 10 (4), 370-373. Pashler, H., & Carrier, M. (1996). Structures, processes, and the flow of information. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 3-29). San Diego: Academic Press. Roediger, H. L., & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 197-236). San Diego: Academic Press. Roediger, H. L., & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (Eds.), Memory (2nd ed. pp. 225-228). San Diego: Academic Press. Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1 (3), 181-210. Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long term retention. Psychological science, 17 (3), 249-255. Rohrer, D., Dedrick, R.F. & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of educational psychology, 107 (3), 900-908. Schellenberg, G.E. & Hallam, S. (2005). Music listening and cognitive abilities in 10- and 11-year-olds: The blur effect. Annals of the New York Academy of Sciences, 1060, 202-209.
Smith, S.M., (1985). Background music and context-dependent memory. The American Journal of Psychology, 98 (4), 342-353. Smith, S.M., Glenberg, A., & Bjork, R.A. (1978). Environmental context and human memory. Memory & Cognition, 6 (4), 342-352. Tulving, E., & Thompson, D. M. (1971). Retrieval processes in recognition memory: Effects of associative context. Journal of experimental psychology, 87 (1), 116-124. Tulving, E., & Thompson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological review, 80 (5), 352-373.