Optimizing computer-based second language vocabulary learning Tatsuya Nakata Kansai University, Japan [email protected]This research was supported in part by Grant-in-Aid for Young Scientists (A) #16H05943 and Grant-in-Aid for Research Activity Start-up (#15H06746) from MEXT.
168
Embed
Optimizing computer-based second language vocabulary learning
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Optimizing computer-based second language vocabulary learning
Abstract• This presentation will examine how we can optimize
computer-based vocabulary learning in a second or foreign language (L2). Specifically, we will discuss empirical studies examining the effects of the following factors on vocabulary learning: studying vs. testing, part vs. whole learning, massed vs. spaced learning, equal vs. expanding spacing, immediate vs. delayed feedback, and single vs. repeated retrievals. These studies can recommend to us the optimal way to learn vocabulary from computer software.
• Flashcard (word card) learning: L2 on one side and meaning (L1 translation or definition) on the other.
Initial presentation
Retrieval
Feedback (Correct answer)
Feedback (Correct answer)
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
Effects of studying and testing
Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of XXX for learning. Science, 319, 966-968.
Method
• Participants: 40 American undergraduate students • Materials: 40 Swahili-English word pairs (e.g.,
chakura- food)• Independent variable: Number of study and test
episodes Study : chakura = food Test : chakura = ??? * No correct answer (e.g., food) was provided as
feedback after each test.
4 Conditions
* No correct answer (e.g., chakura = food) was provided as feedback after each test.
# of Study episodes
# of Test episodes
1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4
Method
• Dependent variable: One-week delayed meaning recall test (e.g., chakura = ???)
• Please rank the following four conditions in order of effectiveness (meaning recall 1 week after learning):
# of Study episodes
# of Test episodes
1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4
* No correct answer (e.g., chakura = food) was provided as feedback after each test.
Repeated study + Repeated test
ControlRepeated test Repeated study
Adapted from Roediger 2009
Discussion• Number of testing, not studying, affects long-term
retention.• Increasing the number of studying from two to four
does not significantly increase retention 1 week after learning ( 33 36%⇒ )
• Increasing the number of testing from two to four, in contrast, significantly increases retention 1 week after learning ( 33 81%⇒ )
• When the number of testing is four, increasing the number of studying from two to four does not affect retention ( 81% 81% ⇒ )
Discussion• Vocabulary learning is a function of the
number of testing, not studying.• People learn more from testing than studying
(testing effect).• Gates, A.I. (1917). Recitation as a factor in
memorizing. Archives of Psychology, 6(40).• Karpicke, J. D., & Roediger, H. L. (2008). The
critical importance of retrieval for learning. Science, 319, 966-968.
• NY Times Jan. 20, 2011
Pedagogical implications
• Vocabulary quizzes are good!• It doesn’t necessarily mean that we have to
give “tests” in its traditional sense.• Self-testing also counts as a test.
(e.g.,) activities involving self-testing such as flashcards, fill-in-the-blanks, matching exercises, or translation exercises are helpful.
• Computer-based flashcards are also useful.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T. (2015). Effects of expanding and equal spacing on second language vocabulary learning: Does gradually increasing spacing increase vocabulary learning? Studies in Second Language Acquisition. 37, 677-711. doi:10.1017/S0272263114000825
Background
• Karpicke and Roediger (2008) showed that testing facilitates vocabulary learning.
• How should the tests be distributed to maximize learning?
• Two practice schedules: equal and expanding spacing
Equal spacing vs. Expanding spacing• Equal spacing: The intervals between
encounters of a given item are held constant.
Note. A = a given target word.
Equal spacing vs. Expanding spacing
• Expanding spacing: The intervals between encounters of a given item are gradually increased as learning proceeds.
Note. A = a given target word.
Massed schedule (no spacing)• Target items are encountered several
times in a row.
E.g.,)A A A A B B B B C C C C D D D D …
• Used as the control treatment.
• Which is more effective for vocabulary learning, equal or expanding spacing?
• Expanding spacing is often regarded as the most effective practice schedule (Baddeley, 1997; Ellis, 1995; Hulstijn, 2001; Landauer & Bjork, 1978; Nation, 2001; Pimsleur, 1967; Schmitt, 2000).
• A common feature among existing flashcard software (e.g., iKnow, Word Engine, SuperMemo; Nakata, 2011)
• Meta-analysis by Cepeda et al.(2006): Mixed results
• Why inconsistent results?The effects of expanding spacing may be affected by at least four factors … (Balota et al., 2007; Dobson, 2012; Karpicke & Roediger, 2007; Logan & Balota, 2008; Maddox et al., 2011; Roediger & Karpicke, 2010; Storm, Bjork, & Storm, 2010)
Empirical evidence• Expanding spacing tends to be effective
when … (1) the task difficulty is high, (2) feedback is not provided after retrieval, (3) the treatment involves a large amount of
spacing, and / or (4) the retention interval is shorter than 24 hours (i.e., posttest is given within 24 hours after the treatment).
Limitations• None of the existing L2 vocabulary
studies manipulated these four factors.(Kang et al., 2014; Karpicke & Bauernschmidt, 2011; Pyc & Rawson, 2007)
• This study will consider the effects of (a) the amount of spacing and (b) retention interval (i.e., timing of posttest).
Research questions1. Do (a) the amount of spacing and (b)
retention interval influence the effects of equal and expanding spacing on L2 vocabulary learning?
2. Which has a larger effect on L2 vocabulary learning: the type of spacing (equal or expanding) or the amount of spacing?
METHOD
• Participants: 128 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, and dally
Independent variables• 1) Amount of spacing (between-participant)
Massed > Short = Medium > Longp < .004, 0.78 < d < 4.75 (medium to large effect)
Note. Brackets enclose +1 SE.
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short = Massed n.s. d = 0.39 (small effect)
Medium = Long > Massedp < .014, 0.80 < d < 0.85 (large effect)
Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Spacing effect
0.0
0.2
0.4
0.6
0.8
1.0
Learning phase performance
Pro
porti
on o
f cor
rect
resp
onse
s
Retrieval 1 Retrieval 2 Retrieval 3
Massed
Short
Medium
Long
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Productive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
Short > Massedp < .001, d = 1.07 (large effect)
Short = Massed n.s. d = 0.39 (small effect)
Spacing-retention interval
interaction
Mas
sed
Sho
rt
Med
ium
Long
Mas
sed
Sho
rt
Med
ium
Long
Receptive posttest
Pro
porti
on o
f cor
rect
resp
onse
s
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0Immediate Delayed
SUMMARY
• 1) Very little difference between equal and expanding spacing.
• 2) Expanding spacing was no more effective than equal spacing regardless of (a) the amount of spacing or (b) retention interval.> Expanding spacing may not be the most effective practice schedule.
SUMMARY
• 3) The massed schedule (no spacing) was the least effective in both short and long term.> The finding underscores the importance of spacing in L2 vocabulary learning (spacing effect).
Summary (2): Amount of spacing• 4) Medium and long spacing were the most
effective in both short and long term.• 5) Short spacing was effective in the short term,
but not in the long term.> Introducing a large amount of spacing may be effective, especially for improving long-term retention (spacing-retention interval interaction).>> The amount of spacing may have a larger effect on learning than the type of spacing.
PEDAGOGICAL IMPLICATIONS• The desirable difficulty framework (e.g., Bjork,
1994, 1999)Slow and effortful learning = good retentionFast and easy learning = poor retention(e.g.,) Massed vs. medium / long spacing
• Do not discourage learners from making errors• Make a distinction between learning phase
performance and long-term retention • We should not be concerned too much about
2. The retrieval effort hypothesis (Karpicke & Bauernschmidt, 2011)
LIMITATIONS1. (a) Task difficulty and (b) feedback were
not manipulated. 2. Short duration of the treatment (approx
25 minutes). Kang, Lindsey, Mozer, & Pashler (2014): Treatment conducted over 4 weeks. No difference between equal and expanding spacing.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T., & Webb, S. (2016). Does studying vocabulary in smaller sets increase learning? The effects of part and whole learning on second language vocabulary acquisition. Studies in Second Language Acquisition, 38, 523-552. doi: 10.1017/S0272263115000236
Part vs. whole learning
• Whole learning: The materials to be learnt are repeated in one large block.
• Part learning: The materials are divided into smaller blocks and repeated.
Part vs. whole learning
• When memorising a poem, would it be more effective to read the whole poem several times (whole learning) or would it be more effective to read part by part for the same number of times (part learning)?
• When learning to play music, would it be effective to practise part by part (part learning) before trying to play the whole tune (whole learning)?
Part vs. whole learning
• When learning 20 words… • Whole learning: 20 words are repeated in one
large block• Part learning: 20 words are divided into
smaller blocks and repeated
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Whole learning: 20 items are repeated in one large block
Part learning: 20 items are repeated in four blocks of 5 items
Question
• Which do you think is more effective for L2 vocabulary learning, whole learning or part learning?
• The retrieval practice effect (Baddeley, 1997; Ellis, 1995) and list-length effect (Gillund & Shiffrin, 1984; Van Bussel, 1994): part learning > whole learning
Limitation of previous research• In existing studies, the part-whole distinction
and spacing have been confounded. (i.e., whole learning always had longer spacing than part learning).
• Spacing effect: Large spacing > short spacing (e.g., Bahrick & Phelps, 1987; Cepeda et al., 2008, 2009; Pashler et al., 2003)
• The results of the earlier studies: due to spacing or the part-whole distinction?
Whole learning
Whole learningSpacing of 19 other items
Part learning
Part learningSpacing of 4 other items
Whole learning: Spacing of 19 items⇒ Large spacingPart learning: Spacing of 4 items⇒ Short spacing
Limitation of previous research
• Spacing effect: Large spacing > short spacing.• The results of the earlier studies: due to
spacing or the part-whole distinction?
Present study
• Purpose: investigate the effects of part and whole learning on L2 vocabulary learning while isolating the effects of the part-whole distinction and spacing.
Experiment 1
• Purpose: To investigate the effects of part and whole learning that were matched in spacing.
• RQ: Is whole learning more effective than part learning for L2 vocabulary learning when spacing is equivalent?
METHOD
• Participants: 91 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally
Independent variable
• Unlike previous studies, part and whole learning were matched in spacing (approximately 4 min).
Part / whole distinction Block size
(1) 4-item part Part 4(2) 10-item part Part 10(3) Whole Whole 20
• (1) Productive test: Translate from Japanese (L1) to English (L2).
Pre / Posttests
• (2) Receptive test: Translate from English (L2) to Japanese (L1).
Pre / Posttests
RESULTS
F (2, 88) = 0.14, p = .867, ηp
2 < .01
F (2, 88) = 0.41, p = .664, ηp
2 = .01
Discussion• Previous studies: Whole > part learning• This study: Whole part learning≒• As long as spacing is equivalent, the part-
whole distinction has little effect on learning.• The results of the earlier studies may have
been due to spacing rather than the part-whole distinction?
Limitation of Experiment 1
• We cannot necessarily attribute the lack of statistical significance to the fact that part and whole learning were controlled for spacing (e.g., small sample size, Type II error).
Experiment 2• Purpose: Test the three hypotheses below:• H1: When spacing is equivalent, whole learning does
not outperform part learning• H2: When whole learning has longer spacing than part
learning, whole learning outperforms part learning.• H3: Part learning with longer spacing outperforms part
learning with shorter spacing. • If all three hypotheses are supported, the lack of
statistical significance in Experiment 1 was perhaps because part and whole learning had equivalent spacing.
METHOD
• Participants: 78 Japanese college students
• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally
Independent variable
* (2) “4-item part” and (3) “Whole” learning will be collectively referred to as the experimental treatments.
Part / whole distinction Block size
Spacing(Approx.)
(1) Control Part 4 30 sec(2) 4-item part Part 4 3 min(3) Whole Whole 20 3 min
n.s. d < 0.49
RESULTSPart = Whole > Control
0.87 < d < 1.02
n.s. d < 0.49
Productive posttest
• Immediate: Experimental = Control • Delayed: Experimental > Control • Spacing-by-retention interval interaction:
- When the posttest is given after a short delay, short spacing tends to be effective.- When the posttest is given after a long delay, long spacing tends to be effective. (e.g., Cepeda et al., 2006, 2008, 2009; Pashler et al., 2007; Rohrer & Pashler, 2007)
n.s. d < 0.60
Receptive posttest
• Immediate: Experimental = Control • Delayed: Experimental = Control • At each test administration, the productive
test was given prior to the receptive test.• This perhaps affected performance on the
receptive test and reduced a potential difference among the three groups on the receptive posttest?
DISCUSSION• All three hypotheses were supported:• H1: When spacing is equivalent, whole learning
does not outperform part learning (whole 4-≒item part).
• H2: When whole learning has longer spacing than part learning, whole learning outperforms part learning (whole > control).
• H3: Part learning with longer spacing outperforms part learning with shorter spacing (4-item part > control).
Experiments 1 & 2
• As long as spacing is equivalent, the part-whole distinction has little effect on learning
• Spacing has a larger effect on learning than the part-whole distinction
Pedagogical implications
1. As long as the treatment involves long spacing, learners may study with either part or whole learning without significantly affecting learning outcomes.
2. It is useful to pay more attention to spacing rather than the part-whole distinction. The amount of spacing has a larger effect on learning than the part-whole distinction.
Limitations
• 1. Short duration of the treatment (< 20 minutes).
• 2. Learning was restricted to flashcard learning.
Outline
1. Effects of studying and testing (Karpicke & Roediger, 2008)
2. Effects of equal and expanding spacing (Nakata, 2015)
3. Effects of part and whole learning (Nakata & Webb, 2016)
4. Effects of single and repeated retrievals (Nakata, in press)
5. Q&A
• Nakata, T. (in press). Does repeated practice make perfect? The effects of within-session repeated retrieval on second language vocabulary learning. Studies in Second Language Acquisition.
Arguments against within-session repeated retrieval
• Retrieval effort hypothesis: concerned with the issue of efficiency (i.e., whether repeated retrieval allows learners to learn a large number of words when time-on-task is controlled).
• Difficult and effortful retrievals > easy and effortless retrievals
• In within-session repeated retrieval, retrievals will be progressively easier (e.g.,) 1st < 2nd < 3rd
• Most learning occurs in the first few retrieval attempts, and practicing retrieval beyond a certain frequency level may not provide a benefit that justifies its cost (i.e., increased study time).
• Within-session repeated retrieval is not necessarily efficient?
1. Examine both the short and long term effects of within-session repeated retrieval.Posttests were given not only immediately and 1 week, but also 4 weeks after the treatment (Rohrer et al., 2005).
2. Examine not only effectiveness (posttest scores) but also efficiency (posttest scores / study time).
Research questions
• RQ1: In L2 vocabulary learning, does increasing within-session retrieval frequency increase effectiveness (posttest scores)?
• RQ2: In L2 vocabulary learning, does increasing within-session retrieval frequency increase efficiency (posttest scores / study time)?
Hypotheses
• Hypothesis 1: Within-session repeated retrieval is more effective than fewer retrievals immediately and 1 week after learning, but not 4 weeks (spacing effect).
• Hypothesis 2: Within-session repeated retrieval is more efficient than fewer retrievals regardless of the posttest timing (retrieval effort hypothesis).
METHOD
• Participants: 98 Japanese college students
• Target items: 16 low frequency English wordse.g.,) apparition, billow, citadel, dally, gouge
METHOD
• Participants: 98 Japanese college students
• Target items: 16 low frequency English wordse.g.,) apparition, billow, citadel, dally, gouge
Independent variable
• (1) Retrieval frequency (between-participant): 1, 3, 5, or 7 retrievals
• (2) Posttest timing (within-participant): immediately, 1 week, and 4 weeks after learning
Procedure
Week 1Pretest (a. Productive b. Receptive)Treatment Filler taskImmediate posttest (a. Productive b. Receptive)
Week 2 Delayed posttest 1 (a. Productive b. Receptive)
Week 5 Delayed posttest 2 (a. Productive b. Receptive)
1st Encounter: Initial presentation
Retrieval opportunities (1, 3, 5 or 7 times)
Feedback (Correct answer)
• (1) Productive test: Translate from Japanese (L1) to English (L2).
Pre / Posttests
• (2) Receptive test: Translate from English (L2) to Japanese (L1).
• Hypothesis 1: benefits of within-session repeated retrievals are short-lived
• This study: repeated retrievals > fewer retrievals regardless of posttest timing
• The negative effects of massing practice were outweighed by the positive effects of increased frequency?
• Underscore the importance of repetition in L2 vocabulary learning.
Discussion: Efficiency
• Single retrieval was the most efficient regardless of the posttest timing, supporting Hypothesis 2.
• The findings are consistent with the retrieval effort hypothesis, according to which the positive effects of repeated retrieval decrease as a function of the retrieval frequency.
• Most learning occurred in the first retrieval attempt, and practicing retrieval more than once did not provide a benefit that justified its cost (i.e., increased study time).
Pedagogical implications
• If efficiency is not an issue, increasing retrieval frequency within one learning session is preferable.
• If learners do not have enough time, it may be desirable to practice retrieval only once.
Limitations
1. The posttest timing was manipulated within participants.
2. The treatment in this study was restricted to flashcard learning.
SUMMARY
• Karpicke and Roediger (2008)• People learn more from testing than studying
(testing effect).• How should the tests be distributed to
maximize learning?• Two practice schedules: equal and expanding
spacing
Equal and expanding spacing
• 1) Very little difference between equal and expanding spacing.
• 2) The massed schedule was the least effective in both short and long term (spacing effect).
• 3) Medium and long spacing were the most effective in both short and long term.
• 4) Short spacing was effective in the short term, but not in the long term (spacing-retention interval interaction).
Whole and Part Learning
• As long as spacing is equivalent, the part-whole distinction has little effect on learning.
• Spacing has a larger effect on learning than the part-whole distinction.
• It is useful to pay more attention to spacing rather than the part-whole distinction.
Repeated retrieval• If efficiency is not an issue, increasing retrieval
frequency within one learning session is preferable.
• If learners do not have enough time, it may be desirable to practice retrieval only once.
• I am very grateful to Stuart Webb, Paul Nation, Jan Hulstijn, Rod Ellis, Stuart McLean, Atsushi Mizumoto, and Tomohiro Tsuchiya.