Sachs_Polio_2007.pdf

LEARNERS’ USES OF TWOTYPES OF WRITTEN

FEEDBACK ON A L2 WRITINGREVISION TASK

Rebecca SachsGeorgetown University

Charlene PolioMichigan State University

This study examines the effectiveness of written error correctionsversus reformulations of second language learners’ writing as twomeans of improving learners’ grammatical accuracy on a three-stagecomposition-comparison-revision task. Concurrent verbal protocolswere employed during the comparison stage in order to study thelearners’ reported awareness of the more targetlike reformulations.The reactivity of think-alouds as a research tool was also assessed.First, 15 adult learners of English participated in a repeated-measuresstudy with three experimental conditions: error correction, reformula-tion, and reformulation + think-aloud. Participant reports of aware-ness in the reformulation + think-aloud condition suggested thatnoticing of feedback was related to the accuracy of subsequent revi-sions. A second nonrepeated-measures study was then carried outwith 54 participants; a control group was added and the design wasmodified in an attempt to eliminate the reported tendency of learnersto develop and use memorization strategies while processing the writ-ten feedback. In both experiments, participants performed signifi-cantly better in the error correction condition than in the reformulationcondition. The think-alouds, used to examine learners’ attentional pro-cesses, were found to be reactive in the first study; learners in thereformulation condition produced significantly more accurate revi-sions than those who were asked to think aloud while processing the

We would like to thank Alison Mackey and the anonymous SSLA reviewers for their helpful sugges-tions on this article+

Address correspondence to: Rebecca Sachs, Georgetown University, Department of Linguistics,Intercultural Center, 4th floor,Washington, DC 20057; e-mail: rrs8@georgetown+edu; or Charlene Polio,Michigan State University, Department of Linguistics,Wells Hall, A714, East Lansing, MI 48824; e-mail:polio@msu+edu+

SSLA, 29, 67–100+ Printed in the United States of America+DOI: 10+10170S0272263107070039

© 2007 Cambridge University Press 0272-2631007 $12+00 67

reformulations they received. The results suggest that whereas ver-bal protocols might be able to shed some light on learner-internalprocesses in relation to written feedback, they should be employedand interpreted with care.

FOCUS ON FORM AND WRITTEN FEEDBACK

A good deal of research in the field of SLA supports the idea that second lan-guage ~L2! learners benefit from explicit or implicit means of drawing theirattention to linguistic form+ Schmidt ~2001!, in particular, emphasized the impor-tance of attention for SLA, proposing the working hypothesis that “SLA is largelydriven by what learners pay attention to and notice in target language inputand what they understand the significance of noticed input to be” ~pp+ 3–4!+He asserted that given the communicative redundancy, infrequency, and lackof salience of much L2 input data, “intentionally focused attention may be apractical ~though not theoretical! necessity for language learning” ~p+ 23!+ Insupport of these ideas, Norris and Ortega’s ~2000! meta-analysis of experimen-tal studies, which compared explicit and implicit instructional approaches,found that the explicit approaches tended to be more effective; the term explicit~following DeKeyser, 1995! was used to signify either that a rule was providedor that learners were directed to attend to a specific form+ R+ Ellis ~2001! like-wise noted that form-focused instruction is associated with higher learningrates and ultimate achievement+ Furthermore, according to N+ Ellis ~2005!,explicit knowledge and attention to form can have an impact on implicit lan-guage learning through, for example, influencing learners’ processing and useof language cues, helping them to consciously construct utterances that thenserve as linguistic input+

Several studies also found that recasts, commonly classified as implicit neg-ative feedback, can be effective ~e+g+, Braidi, 2002; Iwashita, 2003; Leeman, 2003;Mackey & Philp, 1998; Philp, 2003; see Long, 1996, 2007, and Nicholas, Light-bown, & Spada, 2001, for reviews!; however, researchers increasingly pointout that their effectiveness appears to be greater when learners are given cuesto their corrective nature ~e+g+, Lyster, 1998; Nicholas et al+, 2001; Roberts, 1995!+Recent work by Loewen and Philp ~2006!, for instance, suggested that thedegree of explicitness of a recast, the clarity of its purpose, and its compara-bility with a learner’s original utterance ~e+g+, by containing prosodic empha-sis, relatively few changes, and shortened length! can affect learner uptakeand subsequent accuracy of use ~see also Egi, 2004; Philp, 2003; Sheen, 2004!+More generally, Carroll ~2001! contended that in order for learners to restruc-ture their mental representations as a result of feedback, they must construecorrections as metalinguistic commentaries on their production and attendspecifically to relevant aspects of the corrections+ Together, these proposalsand findings point to the likelihood that the means of encouraging learners tofocus on form might be facilitative of L2 development+

Researchers of L2 writing, such as Cumming ~1990!, Polio, Fleck, and Leder~1998!, Qi and Lapkin ~2001!, and Swain ~1998! have claimed that the written

68 Rebecca Sachs and Charlene Polio

modality can provide learners with special opportunities to focus on linguis-tic form+ Cumming ~1990! argued, for instance, that the “natural disjuncturebetween written text and the mental processes of generating and assessing it”might help learners pay attention to form-meaning relationships ~p+ 483!+ Aslearners monitor and refine their linguistic expression, they might gain morecontrol over their language knowledge and production processes, an opportu-nity that might not be as possible in speech, given its fleeting nature and theexigencies of real-time conversation+ In fact, whereas VanPatten ~1990! foundthat it was difficult for learners to attend to both form and meaning at the sametime in spoken language, Wong ~2001! showed that learners could accomplishthis in the written modality+ The juxtaposition of nontargetlike forms with cor-rect alternatives, which Saxton ~1997! highlighted as valuable in his contrasttheory of negative input in the context of oral interactions,might be especiallyuseful in the written mode, given the relative concreteness of corrections thereand the less pressing time constraints that obtain+ Indeed, researchers inves-tigating the effectiveness of written recasts have found that they promotelinguistic accuracy when provided by an interlocutor in synchronous computer-mediated communication ~Choi, 2000; Sachs & Suh, in press!+ With theseconsiderations in mind, it seems that written feedback should provide oppor-tunities for learners to focus their attention on form, perhaps making cogni-tive comparisons ~R+ Ellis, 1994! and noticing the gap ~Gass, 1997; Schmidt &Frota, 1986! between their interlanguage ~IL! and the target language ~TL!+

Unfortunately, no empirical evidence has shown that error corrections onlearners’ written compositions in fact facilitate language acquisition+ Trus-cott’s ~1996! review article of error correction proposed various theoreticaland practical problems related to its effectiveness+ He claimed that grammaracquisition is not a sudden discovery and that the memorization of explicitrules or corrections is superficial and transient, even going so far as to statethat grammar correction is counterproductive and should therefore be aban-doned+ Although many ~e+g+, Chandler, 2003; Ferris, 1999, 2004! have arguedthat Truscott overstated the findings that support his thesis, it is true thatthe effectiveness of error correction has not been demonstrated in the con-text of L2 writing+1 Furthermore, one cannot prove a negative hypothesis ~i+e+,that error correction does not work!, and so the effectiveness of written errorcorrection remains an open question+

REFORMULATION AS AN ALTERNATIVETO ERROR CORRECTION

Even among those who assume that written feedback is helpful, the form thaterror correction should take has been debated+ Some early researchers sug-gested, for instance, that teachers should provide cues instead of explicitcorrections, thereby encouraging students to become actively engaged in pro-cessing feedback as a problem-solving activity ~e+g+, Makino, 1993!+ When Robb,Ross, and Shortreed ~1986! compared four different methods of providing feed-

Learners’ Uses of Written Feedback 69

back on errors in students’ writing along these lines, however, they found nodifferences in long-term effectiveness with respect to linguistic accuracy+2 Morerecently, Qi and Lapkin ~2001! argued that written error correction “does notprovide optimal conditions to help learners notice their errors, i+e+, the gapbetween their IL and TL when they receive and process the feedback” ~p+ 280!+As an alternative, they suggested the use of reformulation, defined by Thorn-bury ~1997! as a native speaker’s reworking of an L2 learner’s written com-position in order to make the language seem as nativelike as possible whilekeeping the content of the original intact+

One of the stated bases for Qi and Lapkin’s ~2001! suggestion is Swain’s~1985, 1995; Swain & Lapkin, 1995! output hypothesis, which proposes thatthe struggles learners go through when they attempt to convey a particularmessage might promote awareness of holes in their linguistic knowledge andinduce them to notice relevant language forms in future input, thereby possi-bly influencing what becomes intake and what is learned+ One could arguethat error correction should achieve the same goals, but Qi and Lapkin sug-gested that a reformulation’s rich and “positive modeling of native-like writ-ing may be more helpful to the learner than error correction” ~p+ 295!+ It isalso possible to speculate, as previous researchers have, that active engage-ment in attempting to identify and understand errors might be helpful as well+Turning to another strand of research, support for the potential benefits ofreformulation might come from the literature on lexical acquisition+ Laufer andHulstijn ~2001!, for instance, discussed the differences between looking upunfamiliar words in a dictionary versus being presented with their definitionsor translations in marginal glosses while reading+ If Laufer and Hulstijn’s ideasregarding search and depth of processing can be extended beyond the area ofvocabulary learning and if learners tend to engage more actively with refor-mulations as a form of corrective feedback, then, hypothetically speaking, learn-ers’ approaches to reformulations might tend to promote L2 development+ Inrelation to written feedback, it seems clear that searching one’s own knowl-edge of language for an understanding of grammatical differences is not thesame as searching for a word in a dictionary, which at least can be expectedto provide a relatively complete and unambiguous answer+ However, the ideahere would be that if learners must engage their IL systems in searching formismatches and must evaluate or question their existing knowledge while theydeal with feedback, then this—along with any rehearsal processes that mightfacilitate incorporation—might lead to greater uptake than the ~possibly auto-matic or mechanical! noting and copying of overt corrections+

In discussing the importance of making cognitive comparisons, R+ Ellis ~1994!highlighted the notion that it is helpful for learners to notice both differencesand similarities between IL and TL—that is, not only reevaluating their knowl-edge but also confirming it+ With reformulations ~as has also been argued fororal recasts!, given that learners already know the intended meanings of theiroriginal language productions, their cognitive comparisons might be an espe-cially useful way of making form-function mappings+ Of course, this presumesthat learners are able to recognize reformulations as more targetlike versions


of their writing, and—given the similarity between the definition of reformu-lation presented here and that of recasts in interaction research—it is impor-tant to acknowledge that questions have been raised regarding learners’ abilityto identify and make effective use of recasts ~e+g+, Lyster, 1998, 2004; Lyster &Ranta, 1997!+ In fact, using stimulated-recall methodology, Mackey, Gass, andMcDonough ~2000! found that learners did not tend to perceive oral recastsof their morphosyntactic errors as corrections+ However, in light of the find-ing that context exerts an influence on learners’ uptake of feedback ~e+g+, Sheen,2004!, it seems plausible that learners who know their writing has been refor-mulated might be oriented toward looking for corrections+ With both errorcorrections and reformulations or recasts, learners are provided with the nec-essary repairs ~Lyster, 2004!; it is possible to speculate, though, that thesetypes of feedback might differ in terms of how learners approach and engagewith their different formats+

LEVELS OF REPORTED AWARENESS

In Qi and Lapkin’s ~2001! pilot study of reformulated L2 writing, two Mandarin-speaking learners of English engaged in a three-stage writing task ~drafting awritten picture description, comparing it with a reformulated version, and revis-ing it!, thinking aloud throughout the process+ The researchers used the learn-ers’ verbalizations to claim that noticing can be either “perfunctory ~i+e+,noticing only and without giving reasons! or substantive ~i+e+, noticing andproviding reasons!” ~p+ 291! and that the depth of this processing was directlyrelated to improvement on revisions+ The statements of causality made in Qiand Lapkin’s article ~e+g+, that “noticing without understanding or noticing forno articulated reason does not have the same impact on learning in L2 per-formance as does noticing with understanding” @p+ 294# ! require further dis-cussion+ First, however, it is important to point out some parallels between Qiand Lapkin’s distinction and Schmidt’s ~2001! differentiation between two lev-els of awareness—already fairly well established and empirically supportedin the literature on attentional processes—namely awareness at the level ofnoticing and awareness at the level of understanding ~Leow, 1997, 2001; Rosa& Leow, 2004; Rosa & O’Neill, 1999!+3

There are at least two major positions on attentional processes in SLA, whichdiffer in their approaches to the role of awareness+ Schmidt ~1990, 1995, 2001!and Robinson ~1995, 2001! believe that awareness is necessary for learning,whereas Tomlin and Villa ~1994! do not+ The noticing referred to in Schmidt’snoticing hypothesis means the conscious perception of surface-level linguis-tic phenomena or concrete “input as interpreted by existing schemata” ~2001,p+ 31!+ It is awareness at this level of noticing that is required for acquisition,whereas awareness at the higher level of understanding ~involving, e+g+, abstractrules, hypotheses, or comparisons! might be facilitative of acquisition but isnot necessary+ In Schmidt’s view, there is no such thing as subliminal languagelearning; unnoticed stimuli pass out of short-term memory within a few sec-


onds and are not committed to long-term storage+ Robinson ~1995! similarlysees noticing as a necessary first step in language acquisition and defines itas “detection plus rehearsal in short-term memory, prior to encoding in long-term memory” ~p+ 296!+ In Tomlin and Villa’s fine-grained model of attention,on the other hand, there are three attentional functions: alertness ~readinessto deal with incoming stimuli!, orientation ~direction of attentional resourcestoward stimuli!, and detection ~cognitive registration of stimuli!, the last ofwhich is argued to be necessary and sufficient for SLA, even without aware-ness+ Alertness and orientation, for their part, might serve to make detectionmore likely ~and might be promoted by certain focus-on-form instructional tech-niques!, but they do not necessarily ensure detection or learning+

Leaving aside the details of this debate, which involves making thorny dis-tinctions between detection and noticing, we note that empirical research hasin fact shown higher levels of reported awareness to be more strongly asso-ciated with learning, consistent with Qi and Lapkin’s ~2001! claims+ In a studythat examined the think-aloud protocols of beginning learners of Spanishengaged in a crossword puzzle task, for example, Leow ~1997! demonstratedthat characteristics of their verbalizations were related to their linguistic accu-racy on subsequent written recognition and production tasks; learners whomade metacomments and stated rules about the targeted L2 forms performedsignificantly better than did learners who simply mentioned the forms+ Like-wise, Rosa and O’Neill ~1999! and Rosa and Leow ~2004! showed that whereasreports of noticing targeted forms were associated with intake, verbalizationsof higher levels of awareness with evidence of understanding were associatedwith greater intake+

Notably, a large part of the hypothesized foundation for the effectivenessof reformulation is based on learner-internal factors, such as attentional pro-cesses, approaches to feedback, and cognitive comparisons+ In relation to this,Leow ~1999! argued that any studies whose rationales are based on the impor-tance of attention should attempt to measure it directly; otherwise, only indi-rect inferences can be made+ Verbal protocols have proven to be a usefulresearch tool for this purpose; however, as will be discussed in the next sec-tion, they have certain limitations in addition to their demonstrated benefits+

VERBAL PROTOCOLS

Because it is not possible to report on something unless one is aware of it,verbal reports have been put forward as the “clearest evidence that some-thing has exceeded the subjective threshold and been consciously perceivedor noticed” ~Schmidt, 2001, p+ 20!+ As mentioned previously, by looking morecarefully at the content of verbalizations produced by learners engaged inlanguage-related tasks, researchers have made inferences regarding theirlevels of awareness with respect to particular linguistic targets ~e+g+, Leow,1997, 2001; Rosa & Leow, 2004; Rosa & O’Neill, 1999; Sachs & Suh, in press!+It would be foolhardy to assume that learners verbalize everything that they


notice ~Jourdenais, 2001!; nonetheless, introspective measures do provide moreinsight than can be gleaned from simply watching learners perform tasksand are increasingly used to investigate learner-internal processes in SLAresearch+

There are important issues to keep in mind when concurrent verbaliza-tions are employed as a research tool, however—most notably, concerns aboutreactivity and nonveridicality+ Ericsson and Simon ~1993! asserted that verbalprotocols do not interfere with cognitive processes as long as participantssimply report on the contents of short-term memory+ Concerns have beenraised in SLA research, however, that metalinguistic verbalizations in particu-lar might have an impact on task completion+ In fact, when used along with afeedback processing task, talking aloud might create extra learning opportu-nities, promote increased attention, and lead to deeper processing, more rea-soning, and, ultimately, better revisions ~cf+ Jourdenais, 2001; Stratman & Hamp-Lyons, 1994!+ In Qi and Lapkin’s ~2001! study, for instance, the act of thinkingaloud might actually have enhanced or caused some of the noticing thatoccurred+ Alternatively, the additional cognitive load might have distractedlearners or decreased what they noticed+ It is also possible that think-aloudsmight incompletely or inaccurately reflect participants’ thinking processes~Jourdenais!+ There is no certain way of knowing whether a given verbaliza-tion is a veridical ~i+e+, complete and accurate! account of a learner’s aware-ness of linguistic input, which makes relationships between awareness andother phenomena difficult to determine with confidence+

In a study by Leow and Morgan-Short ~2004! that investigated the issue ofreactivity with beginning Spanish learners performing a L2 reading task, think-alouds were not found to be reactive+ However, a possible explanation for this,according to the researchers, might have to do with the fact that translationappeared to be many learners’ preferred reading strategy+ A more recent studyby Bowles and Leow ~2005!, which built on this work, attempted to inducehigher proficiency L2 learners to produce either metalinguistic or nonmetalin-guistic think-alouds through instructions that requested specific kinds of ver-bal reports+ They did not find concurrent verbalization to be reactive for eitherof these experimental groups when compared to a silent control ~except interms of time on task!; however, the learners who produced nonmetalinguis-tic think-alouds showed significantly better text comprehension than thosewho produced metalinguistic think-alouds, which seems to indicate that thelearners’ reading processes were in fact affected+ Qualitative data from thethink-aloud transcripts provided support for this in that some learners com-mented on the difficulty of verbalizing their thoughts and justifications whilereading+ As discussed previously, it is important for researchers to use onlinemeasures to investigate learners’ attentional processes directly ~Leow, 1999!,and this is particularly important in the context of the current study due toits focus on learners’ awareness and uses of written feedback+ Nonetheless,given that so few published studies have addressed the issue of reactivityempirically in the field of SLA, it is not possible to assume that think-aloudsare a nonreactive methodological tool on all types of L2 task+


RESEARCH QUESTIONS

There were three main areas that we wanted to investigate in this study+ First,because reformulation has been proposed as an alternative form of feedbackon L2 writing that might promote active engagement, search, analysis, aware-ness of positive input, and form-function mapping on the part of learners, wedecided to compare the effectiveness of reformulations versus written errorcorrections in a controlled three-stage composition-comparison-revision task+Second, because the hypothesized foundations for the use of reformulationinvolve learners’ attentional processes and because higher levels of aware-ness have been associated with more accurate subsequent language use onother types of language-related tasks ~e+g+, Leow, 1997, 2001; Rosa & Leow, 2004;Rosa & O’Neill, 1999; Sachs & Suh, in press!, we wanted to test Qi and Lap-kin’s ~2001! claim regarding the relationship between higher quality noticingand revision outcomes by examining the verbal reports made by a subset oflearners who performed the reformulation task+ Last was the methodologicalissue of reactivity, an important one given that the validity of verbal proto-cols as a research tool has not yet been sufficiently established in the field ofSLA+ The research questions were as follows:

1+ Is there any difference in the relative linguistic accuracy of revisions producedwhen students have been given reformulations of their writing versus when theyhave been given written error corrections?

2+ How are learners’ reports of awareness of language related to the changes theymake in revisions completed after they have compared their writing with reformu-lated versions?

3+ Is there any difference in the relative linguistic accuracy of revisions producedwhen students have compared their writing with reformulated versions while simul-taneously thinking aloud versus when they have done so without thinking aloud?

To address these questions, a repeated-measures study ~described in greaterdetail in the Design Overview section! was conducted with three conditions:error correction, reformulation, and reformulation � think-aloud+ The firstresearch question led us to examine the differences in revision outcomes be-tween the error correction and reformulation conditions+ For the secondquestion, we investigated the reported awareness of the participants in thereformulation � think-aloud condition in relation to their linguistic accuracyon subsequent revisions+ To address the third question, we compared the refor-mulation and reformulation � think-aloud conditions with respect to revisionoutcomes+ Because our original interest was in what learners would noticeabout feedback when it was not provided overtly in the form of written cor-rections, we chose not to include an error correction � think-aloud condition+This also means that we investigated the issue of reactivity only with respectto the reformulation condition+ A second nonrepeated-measures study, whichaddressed the first and third research questions with the addition of a con-trol group, will also be discussed+


It should be noted from the outset that in the two studies described in thisarticle, we examine only short-term changes in linguistic accuracy+ Althoughit is ultimately long-term developmental changes that matter, it might, none-theless, be useful here to consider Norris and Ortega’s ~2003! views on defin-ing and measuring acquisition+ In relation to interactionist approaches to SLA,they stated that the term acquired might refer to various sorts of gradual andnonlinear changes in both linguistic and metalinguistic behavior, which includenot only the appropriate use of linguistic forms but also, for example, the con-structs of emergence, detection, restructuring, and awareness+ If we can con-sider this line of thinking to be relevant to studies of written feedback as well,then a range of psychological processes might be seen as constituting stepstoward L2 development, and it might be possible to talk about the process ofacquiring L2 forms based on learners’ verbalizations and revision changes,regardless of what causes what+

STUDY 1: REPEATED-MEASURES DESIGN

Participants

Fifteen students ~11 Korean, 3 Japanese, and 1 Indonesian! enrolled in a high-intermediate English as a second language ~ESL! class in the intensive Englishprogram ~IEP! at a large Midwestern university participated in this study+ Par-ticipants had been in the United States for a range of 1 month to 1 year, andmost of them had recently been placed in the high-intermediate level basedon the IEP placement test, although some had started in the intermediate leveljust below that at the beginning of the previous semester+ Most of the learn-ers were in their early twenties; they ranged in age from 18 to 30+ Most par-ticipants were working toward undergraduate degrees in a range of fields,although a few had already completed their undergraduate studies and hopedto go on to graduate work in the United States+ The female-to-male ratio wasalmost even, with eight females and seven males+

Design Overview

A 3-day sequence of composition, comparison, and revision was performedthree times over the course of 3 weeks, as illustrated in Table 1+ This formatwas developed to investigate the learners’ ability to revise accurately in threedifferent conditions of written feedback: ~a! when given written error correc-tions of their writing, ~b! when given native-speaker reformulations, and ~c!when given such reformulations and asked to think out loud about them+ Eachof the 15 learners whose data are included in this study completed all of thetasks for each condition once, and the only difference among the conditionstook place on the second day of the sequence, during the comparison stage+Condition and writing task were counterbalanced to control for ordering andtask effects+ If a student was absent at any point over the course of the study,


Table 1. Three-day sequences of the three experimental conditions

Condition Tuesday ~30 min! Thursday ~15 min! Friday ~20 min!

Error correction Write a 30-min picturedescription+

Look at written error corrections of thestory+

Revise a clean copy of the original storywithout access to the corrections+

Reformulation Write a 30-min picturedescription+

Compare the story to a reformulatedversion+

Revise a clean copy of the original storywithout access to the reformulation+

Reformulation � think-aloud Write a 30-min picturedescription+

Compare the story to a reformulatedversion while thinking aloud+

Revise a clean copy of the original storywithout access to the reformulation+

76R

ebeccaS

achsand

Charlene

Polio

his or her data were discarded+ For 3 weeks, each participant wrote a storybased on a picture prompt for 30 min on Tuesday ~see Appendix A for anexample!, compared his or her writing with a corrected or reformulated ver-sion for 15 min on Thursday, and then revised the original story for 20 min onFriday without access to the corrected or reformulated version+ Whereas thestudy did take place in a classroom setting, it is important to acknowledgethat many of the controls that were used to enable the comparison of differ-ent experimental conditions also served to reduce direct applicability to theclassroom and, in some cases, represented departures from how the tech-nique of reformulation has traditionally been motivated pedagogically ~i+e+, asa means of providing feedback at higher discourse levels beyond discretepoints of morphosyntax!+ The students were told that they could take noteson the feedback they were given but they would not be able to look at theirnotes or corrections when they rewrote their stories the next day; all of theirmaterials were collected at the end of each session+

Procedure and Coding

To ensure that the participants worked through problems with linguistic out-put on their own, they were not allowed to consult with each other or to usedictionaries while writing, comparing, or revising+ At the end of the 30-mincomposing stage, the teacher ~who was also one of the researchers! col-lected all of the stories along with the pictures+ Each story was then typed,and the errors were coded by both researchers according to a 40-categoryclassification system adapted from Polio ~1997, in turn adapted from Kroll,1990!+ The level of interrater agreement was 83+1% ~simple percentage agree-ment!, slightly higher than the reliability found in Polio+ Each of the disagree-ments was discussed until a consensus was reached, and the agreed-uponcoding was included in the data analysis+ The researchers also jointly iden-tified some expressions that they agreed were not technically incorrect interms of grammar but were unlikely to be used by a native speaker+ Thesewere also marked for correction+ Accidental oversights of unambiguous errors~such as faulty subject-verb agreement! were not counted as disagreements+

After the error coding was completed for all of the students’ stories, one ofthe researchers used the agreed-upon coding to prepare the written feed-back+ For the error correction condition, extra typed copies of the partici-pants’ original stories were made, and corrections ~i+e+, not error codes butactual corrections! were written directly on those sheets in purple ink+ Forthe reformulation and reformulation � think-aloud conditions, the researchertyped reformulations of the learners’ original stories and printed them out onseparate sheets of paper+ Thus, in all three conditions, each learner receiveda typed copy of his or her original story plus a typed copy with either writtenerror corrections or incorporated reformulations+

To ensure that the same kinds of changes would be made in all of the con-ditions, the corrections and reformulations were based specifically and solely


on the errors that had already been coded and agreed upon, with the pur-pose of reworking instances of perceived linguistic inaccuracy, ambiguity, andawkwardness+ As such, we corrected grammatical errors ~choice of preposi-tion, gerund vs+ infinitive, subject-verb agreement, punctuation, verb forma-tion, etc+!, tried to improve style and cohesion ~e+g+, by keeping the verbsof the narrative in the same tense, maintaining parallelism, and makingsure that pronoun references were not ambiguous!, and introduced somenew vocabulary in the form of synonyms for words that were already inthe text+ However, we did not add any sentences or significantly change theorder of existing sentences, and we tried—to the extent possible—not tochange the meaning of what each student was trying to express+ In the end,the main difference between the reformulations and error corrections was amatter of presentation and task demands and was not related to the kinds oferrors that were corrected+ One example of an error is given in ~1!, with anexample of the reformulated sentence in ~2! and an illustration of how an errorcorrection would have been done in ~3!+ A longer excerpt can be found inAppendix B+

The participants in the reformulation � think-aloud condition met individ-ually with one of the researchers outside of class on Thursday to comparetheir stories with the reformulated versions and produce verbal protocols+ ~Inclass, while the other participants were engaged in the comparison stage, thereformulation � think-aloud participants were given free time to read novelsthey had chosen for a reading log project+! To help these learners feel com-fortable producing a verbal protocol, they were given the opportunity to prac-tice speaking beforehand about an original and a reformulated version ofanother piece of writing+ This warm-up was not recorded, in the hope that notrecording it would reduce anxiety+ Comments were made by the researcherduring the think-alouds only to explain the instructions, encourage the par-ticipants to keep talking if they had not spoken for a while, and remind themto speak out loud if they happened to be writing without speaking+ Also, somestudents reached the end of their stories after approximately 10–12 min, sothey were notified of the amount of time remaining to them if they wished tocontinue comparing the two versions+ This reminder was given in the class-room context as well+4

In class on Friday, each student was given only a clean typed copy of theoriginal story he or she had written and was asked to revise it for 20 min,completely rewriting the story on a separate sheet of paper+ These revisionswere then collected and typed+ Again, all of the errors were coded separatelyby the researchers and tallied with regard to the number and type of errors


that had occurred in each version+ At this point, all of the participants’ writ-ing, the corrections and reformulations, the error coding, and their verbaliza-tions were put into a column format, an example of which can be seen inAppendix B+ This was done so that the three stages, along with the tran-scripts of the think-alouds ~where applicable!, could be compared directly witheach other, side by side, in order to evaluate changes in accuracy from oneversion to the next and to investigate any relationships between reportedawareness and revision changes+

Each T-unit ~defined as an independent clause and all of its dependentclauses! in the participants’ revisions was coded for changes in linguistic accu-racy ~i+e+, any observable correction or partial change!+ There were four pos-sible codings in this system: ~a! partially changed ~PC!, if at least one error fromthe original T-unit was changed in the direction of the feedback; ~b! completelycorrected ~CC!, if all of the errors from the original T-unit were corrected;~c! completely unchanged ~UC!; or ~d! not applicable ~NA!, if there had beenno errors in the original T-unit or if the T-unit had been added or deleted+ Weconsidered the PC and CC categories to show changes in accuracy, or at leastsome type of restructuring, whereas the UC category showed no evidence ofthis+ Any T-units that were coded NA were subtracted from the total numberof T-units and were not included in the analysis+ Because all of the individualerrors had already been coded, interrater agreement was very high, at over 99%+

The think-aloud transcripts were also coded in order to address the sec-ond research question+ A two-tiered coding system was used to classify theerrors from the learners’ original stories with respect to the verbalizationsproduced about them ~or the lack thereof!+ On the first tier of this system,each original error ~not T-unit! was coded either �N or �N, for whether ornot it was apparently noticed in the think-aloud, and either �C or �C, forwhether or not it was changed in the revision+ Admittedly, on this initial runthrough the data, we coded as �N a variety of instances that might techni-cally fall outside of Schmidt’s ~2001! definition of awareness at the level ofnoticing or Robinson’s ~1995! definition of noticing as detection plus rehearsalin short-term memory+ Some of them might have constituted simple detection~i+e+, the cognitive registration of stimuli!, whereas others demonstrated aware-ness at the level of understanding ~e+g+, providing an explanation!+ The ratio-nale for this method involved the nature of this study as a partial replicationof that of Qi and Lapkin ~2001!, who used the terms perfunctory and substan-tive noticing+ It should also be pointed out that we originally coded nontarget-like changes separately, but these ~accounting for slightly over 20% of theerrors! were included in the �C category because we considered them to beevidence of awareness of a linguistic problem+5 This first tier of coding hadinterrater agreement at over 99%+ On the second tier of our coding system,quality of noticing was assessed by looking back at all of the �N errors andclassifying them according to the kind of comment made about each one in thethink-aloud+ The categories are shown in Table 2, along with an example of eachtype of comment+ This coding system had an 85+3% interrater agreement+


Table 2. Level of awareness ~quality of noticing! categories

Code Category Example

M Mentioned only or read again withspecial emphasis

Oh, looked at, I missed ‘at+’

SP Misspelling And they threat, threated, I knowthis is wrong spell, so, yeah, changeit+ Mmm + + + threatened people inthe bank with their guns+

ML Use of metalanguage without anexplanatory reason

The women were upset, upset, ah,with him+ Upset with him+ I alsoconfused, um, what kind of preposi-tion I have to choose+

O Oversight The women were upset with himbecause they were + + + right! Theywere worried! @laughs# Why I put‘worry’—yeah, right! Worried+ Theywere worried about hurting her, herdog+

RE Reason Oh, right, and+ Yeah, I had to put‘and’ because I want, I want to con-nect two sentences, so I have to + + +use a connecting word+

LN New lexical item Oh, I learned a new vocabulary:‘make out+’ Make out, makeout means about maybe, mm,determine?

LO Old lexical item Um, sometimes in my, in my work-sheet, uh, I wrote down ‘delight-fully,’ but the, the closer meaning is‘cheerfully,’ so I + + + I change, I haveto change ‘delightfully’ to ‘cheer-fully+’

NR Lack of reason Unfortunately + + + unfortunately, itstarted to rain+ Here I don’t knowwhy put the comma+ @laughs# Actu-ally, I, yeah, I don’t know where Ihave to put comma or semicolon+Actually, I’m, I’m every dayconfused+

RJ Rejection of change No examples available, but thiswould have been something like,“No, that’s not what I meant to say+”

WR Wrong reason I think the verb ‘let’ and verb ‘make’is, uh, similar, so I + + + I wrote the‘let+’ ‘Let’ and ‘make’ is, uh, si- samemeanings sometimes, has a samemeanings, but + + + uh, this situation,maybe ‘make’ is, uh, acceptable+

RD Reading the correction aloud


Results of Quantitative Analyses

Research questions 1 ~feedback method! and 3 ~reactivity! were addressedquantitatively by comparing the error correction condition with the refor-mulation condition and by comparing the reformulation condition with thereformulation � think-aloud condition, respectively+ The results for researchquestion 2 ~awareness! are presented in the section on the analysis of thethink-alouds+

For each participant, the total number of T-units in a given revision thatshowed changes in accuracy ~coded PC or CC! was divided by the total num-ber of T-units in which changes in accuracy were possible ~i+e+, those T-unitsthat contained errors in the original version!+ These data, presented accord-ing to condition in Appendix C, allow us to make preliminary comparisons ofthe revision improvements made by each participant on an individual basisas well as of mean percentages across conditions+ As a whole, the partici-pants in the error correction condition made changes in accuracy on 96+4% ofall T-units that originally contained errors+ This percentage is apparently higherthan the 90+0% of T-units with changes in accuracy in the reformulation con-dition, which is, in turn, higher than the 81+4% in the reformulation � think-aloud condition+6

To answer the first research question, regarding the relative effectivenessof error corrections versus reformulations in promoting revision accuracy inthis three-stage writing task, and the third research question, regarding the reac-tivity of thinking aloud, a Friedman test was first performed to compare themean percentages of revision T-units that contained at least one correction orchange across the three conditions+ The result was significant, x2~2, 15!�16+57,p , +001, and the mean rank of each condition was 2+77 for error correction,1+83 for reformulation, and 1+40 for reformulation � think-aloud+

To address the issue of feedback method more specifically, a Wilcoxonsigned rank test was performed to compare the error correction and reformu-lation conditions+ The results, shown in Table 3, indicate a statistically signif-icant difference and a moderate effect size ~h2 � +56!+ To address the issue of

Table 3. Comparison of the error correction and reformulation conditionsusing Wilcoxon signed rank test

Rank Conditions NMeanrank

Sum ofranks Z p h2

Negative Error correction , reformulation 0 0+00 0+00Positive Error correction . reformulation 10 5+50 55+00Tied Reformulation � error correction 5Total 15 �2+805a +005* 0+5620

*p , +05 ~two-tailed test, statistically significant!+aBased on negative ranks+


reactivity, a Wilcoxon signed rank test comparing the reformulation and refor-mulation � think-aloud groups was performed+ The results, shown in Table 4,indicate that the learners in the reformulation condition performed signifi-cantly better, with a weak effect size ~h2 � +28!+ Similar results were obtainedupon considering the percentages of revised T-units in which all of the errorswere corrected or changed ~i+e+, corresponding only to a CC coding!, as pre-sented in Appendix D+ A Friedman test of ranked percentages showed a signif-icant difference among the groups, ordered in the same way+

Analysis and Results of Think-Alouds

An association was found between the noticing that learners demonstratedduring the think-alouds and the changes they made in their revisions the nextday+ If an error was verbalized, then it was more likely to be changed thannot+ If an error was not verbalized, then it was less likely to be changed+ Theconverse is also true: If an error was changed, then it was more likely to havebeen verbalized than not+ A tally indicated that 261 errors that were noticedwere also changed, whereas only 88 errors that were noticed were not changed+Of the errors that were not verbalized, only 78 were changed, whereas 138remained unchanged+ We did not test for statistical significance here becausethe frequency data do not constitute independent observations+

In a preliminary step toward exploring the relationship between level ofawareness ~or quality of noticing! and changes in revisions, two categorieswere chosen as partially representative of awareness at the level of under-standing ~Schmidt, 1995, 2001!, under the assumption that they would alsorepresent the substantive kind of noticing identified by Qi and Lapkin ~2001!:use of metalanguage and provision of a reason+ In our results, it appears thatproviding a reason for a change or using metalanguage about it during thethink-aloud was associated more with making a change in the revision thanwith not making one; of the 100 observed instances of awareness at the levelof understanding, the use of metalanguage and provision of a reason were asso-ciated with 78 changes, whereas only 22 errors were left unchanged when meta-language was used and a reason was provided+ These results appear to confirmQi and Lapkin’s findings+

Table 4. Comparison of the reformulation and reformulation � think-aloudconditions using Wilcoxon signed rank test

Rank Conditions NMeanrank

Sum ofranks Z p h2

Negative Think-aloud , reformulation 10 9+50 95+00Positive Think-aloud . reformulation 5 5+00 25+00Tied Reformulation � think-aloud 0Total 15 �1+988a +047* 0+2823

*p , +05 ~two-tailed test, statistically significant!+aBased on positive ranks+


Rationale for the Second Study

We originally hypothesized, following Qi and Lapkin ~2001!, that the reformu-lation condition might be particularly effective in promoting changes in lin-guistic accuracy on learners’ revisions because of its presumed encouragementof search and evaluation processes+ Finding that participants produced themost accurate revisions in the error correction condition, we reasoned thatthe format of the error corrections might have given the participants enoughtime to use a range of strategies when processing their feedback, includingmemorization and visualization; that is, not having to search for their correc-tions, they likely had more time to spend on trying to remember the differ-ences for the next day’s revision, regardless of whether they had actuallyunderstood them+ To confirm or deny these speculations, poststudy debrief-ings were conducted with six of the participants soon after the first studywas completed+ The questions are given in Appendix E, along with some oftheir answers+

In the poststudy interviews, all six of the students said that the error cor-rection activity had been the easiest and explained that the writing on thepage had made it easier for them to find their mistakes than when given refor-mulations+ They also mentioned having tried to memorize the changes theyhad seen in a variety of ways—for example, by studying and trying to remem-ber entire sentences ~participant M! or by rereading them several times ~par-ticipant S!+ Interestingly, two participants ~F and J! had tried to remember thevisual characteristics of the page, including what the writing had looked likeand where the errors had been located+ One of them ~F! said that, comparedto the reformulations, the written error corrections had been more “impres-sive” ~i+e+, salient or striking! and easier to remember visually+ The other ~J!added that she had wanted to take the time to memorize the corrections whilethinking aloud, but had not been able to do so because she had had to con-centrate her efforts on talking+ She and participant G also mentioned that theyhad counted their errors and had been successful either in remembering thenumber of errors of a certain kind or in remembering the number of errors ina given sentence or overall+

The participants’ statements during the debriefing interviews also indi-cated that the requirement to talk aloud in their L2 divided their cognitiveresources+ In fact, whereas two participants ~G and S! mentioned that the actof verbalization might ultimately help them remember the corrections for alonger period of time, five of the six participants interviewed ~F, I, J, M, and S!stated that it was difficult to talk aloud, and three ~I, J, M! said that finding thewords to express what they wanted to say made it harder to concentrate onthe corrections themselves+ Other recent SLA studies that employed concur-rent think-aloud protocols have had similar findings, even when the learnerswere allowed to speak in their first language ~L1! ~e+g+, Bowles & Leow, 2005;Sachs & Suh, in press!+

With these issues in mind, some design modifications were made for asecond study in an attempt to temper the participants’ apparent inclination


to make use of memorization strategies, which, we reasoned, were likelyencouraged by the experimental nature of the tasks+ First, the repeated-measures design was abandoned, and data were collected from a greater num-ber of participants in a single 3-day sequence+ In view of the fact that theparticipants would complete the sequence only once each, it seemed lesslikely that they would have as much of a chance to recognize the usefulnessof memorization strategies+ Another important change was the inclusion ofmore time in between the comparison and revision stages+ Instead of havingthe participants complete these tasks during the same week, we made surethat a weekend intervened+ Finally, in order to establish how well the partici-pants were able to revise their stories on their own, a control condition wasadded+ These learners completed exactly the same activities as those in theother three conditions, except that during the comparison stage they lookedat their typed, uncorrected stories for 15 min by themselves while the otherparticipants looked at corrections or reformulations+ The learners’ verbaliza-tions of awareness in the second study were not analyzed in detail+

STUDY 2: NONREPEATED-MEASURES DESIGN

Participants

The participants in the second study were 54 ESL students from a variety oflevels+ Most of them were from the IEP and English for academic purposes~EAP! program at the same large Midwestern university, and 10 came from anESL class at a local community college+ Native languages included mostlyKorean and Japanese, but there were also native speakers of Chinese, Portu-guese, Spanish, and French+ None of the students had participated in the firststudy, and they were randomly divided into conditions within each class+ Therewere 12 participants in the error correction condition, 11 in the reformulationcondition, 16 in the reformulation � think-aloud condition, and 15 controls+~The numbers of participants in these groups are unequal because we dis-carded the data of any students who were absent on 1 or more days of thestudy+!

Results

The same procedures of coding and analysis were followed in this experiment+Even with the design modifications that were made, the error correction con-dition still produced the most accurate revisions, with a mean of 87+6% of theT-units showing changes in accuracy+ As expected, the participants in the con-trol condition—who received no feedback on their writing—wrote the leastaccurate revisions; they showed changes in accuracy in 55+2% of their revisedT-units+ The reformulation and reformulation � think-aloud conditions showedchanges in accuracy in 70+5% and 72+9% of their T-units, respectively+


Because of the small sample size and because a normal distribution wasnot assumed, a nonparametric Kruskal-Wallis test was performed to comparethe mean ranks of percentages across the various conditions+ The result wassignificant, x2~3, 54! � 19+676, p , +01, and the mean rank of each conditionwas 42+63 for error correction ~n � 12!, 28+14 for reformulation ~n � 11!, 26+84for think-aloud ~n � 16!, and 15+63 for the control group ~n � 15!+ ApplyingMann-Whitney tests to check for two-tailed significance in the differencesbetween the conditions in order, the difference in mean rank between errorcorrection and reformulation was significant ~p , +05!, and the differencebetween reformulation � think-aloud and control was significant ~p , +01!+However, the difference in mean rank between the reformulation and reformu-lation � think-aloud conditions was not significant ~p � +77!+ Hence, the sec-ond study confirmed the superiority of the error correction group but did notshow a difference between the reformulation and the reformulation � think-aloud groups+ All groups outperformed the control group+

SUMMARY

Despite logistical problems associated with the use of reformulation as a writ-ten feedback technique in the classroom ~e+g+, the time and effort required ofthe teacher!, we were interested in Qi and Lapkin’s ~2001! suggestion thatreformulations—presented as relevant models of nativelike writing—might bea better tool than what they identified as less-than-optimal error corrections+In the end, at least in terms of leading to short-term changes in linguistic accu-racy on revisions in the specific experimental conditions of this study, theresults showed the opposite of what we had anticipated: Reformulations didnot prove to be more helpful than error corrections in this experimental con-text+ The fact that the participants in the reformulation condition outper-formed those in the control group suggests that reformulations are helpful+However, the participants in the error correction condition consistently pro-duced the most accurate revisions at the level of T-units+

As far as the learners’ attentional processes were concerned, their verbal-izations in the reformulation � think-aloud condition allowed us to confirm,as found by Qi and Lapkin ~2001!, that reports of noticing during the process-ing of written feedback were related to subsequent revision changes+ It alsoappeared that the learners’ uses of metalanguage and provision of reasonsfor the reformulations during the comparison stage were associated with theaccuracy of their revisions+ We found that learners often made use of oppor-tunities to focus on form while examining the reformulations they had beengiven; they frequently perceived particular reformulations as instances of cor-rective feedback, became aware of language problems, and searched their ownlinguistic knowledge in order to try to understand IL-TL differences+ Anotherinteresting finding was related to Swain’s ~1985, 1995! output hypothesis andthe notion of noticing the gap+ Even though our participants did not thinkaloud during the composing stage, some of their comments at the compari-


son stage seemed to indicate problems that had arisen when they wrote thefirst draft of their stories+ In other words, the participants appeared at timesto be aware of their own output problems ~e+g+, participant H mentioned, “Oh,actually, I didn’t know about the past verb of smell, so I just write, wrote downthe present verb+”!+

The think-alouds that we used to examine the learners’ attentional pro-cesses provided insight into relationships between awareness and subsequentlinguistic accuracy+ However, they were found to be reactive as a methodolog-ical tool in the first study; that is, learners in the reformulation condition ~whodid not have to produce concurrent verbalizations! produced significantly moreaccurate revisions than those in the reformulation � think-aloud condition ~whowere asked to verbalize their thoughts while processing the feedback!+ Theseresults suggest that although verbal protocols can shed light on learner-internal processes in relation to written feedback, they should be interpretedwith care+ SLA researchers who employ think-aloud protocols should keep inmind that the requirement to verbalize concurrently might affect their results+

DISCUSSION

Some might be tempted to make generalizations about the unexpected find-ing that error corrections led to more accurate revisions than reformulationsdid in the context of this study+ In the interpretation of these results, how-ever, there are a variety of methodological issues and other factors to con-sider+ These include, to name a few, the perceptual salience of the writtenerror corrections ~particularly in relation to the learners’ allocation of cogni-tive resources in a timed situation and the corresponding ease of identifyingfeedback as corrective!, problems in controlling for time on task in SLA stud-ies, the importance and limitations of poststudy debriefings, and difficultiesin assessing the presence and degrees of awareness and acquisition+ Becausewe found these aspects of our study to be the most instructive, the discus-sion will focus mainly on using them to draw implications for research meth-odology and future studies of learners’ processing of written feedback+

First, it is important to note that the error corrections, which were writtenin a familiar way on the students’ papers in purple ink, likely made the feed-back more perceptually salient, indicating the locations of differences moreclearly than was the case in the reformulation conditions+ All of the partici-pants knew that they would soon be revising their stories, but because the par-ticipants in the error correction condition did not have to search for differences~as in the reformulation conditions! or talk about what they were doing ~asin the reformulation � think-aloud condition!, they might have been betterable to devote cognitive resources to understanding and remembering the cor-rections during the 15 min allotted for the task+ Whereas such divisions ofresources were not directly assessed in this study through online measuresallowing comparisons of the error correction and reformulation conditions, theparticipants themselves corroborated this idea in their poststudy interviews


~see Appendix E!+ Additionally, with regard to the reformulation � think-aloudcondition more specifically, Ericsson and Simon’s ~1993! review of research thatemployed think-aloud protocols discussed latency effects or reactivity for time,and recent research in the field of SLA by Bowles and Leow ~2005! found thatboth of the verbalization conditions in their study took significantly longer thanthe control+ In the current study, the extra time that might have been requiredfor the participants to speak their thoughts aloud might also have meant lesstime spent on the main task of comparing their original stories with the refor-mulated versions+

In fact, the time factor is a problem throughout applied linguistics and ped-agogical research that compares different treatments with respect to learningoutcomes+ This problem was addressed by Polio and Gass ~1998! with regardto interaction studies in which the interactive conditions simply take moretime than the noninteractive conditions, and a similar issue is evident in thevocabulary studies reviewed by Laufer and Hulstijn ~2001!: Treatment condi-tions that require deeper processing ~e+g+, search and evaluation processes!take more time+ Unfortunately, simply giving the participants as much time asthey needed would not have solved the problem in the present study, anddoing so might have biased results in the other direction+

As noted previously, especially in the absence of nonreactive online pro-cess measures for all of the experimental conditions, the poststudy debrief-ings were useful because they gave us insight into the participants’ strategiesand allowed us to confirm possible explanations for our unexpected findings+It is important to note, however, that it is still not possible to say whether thetask demands of any of the conditions encouraged deeper processing, moremetalinguistic awareness, the development of cognitive strategies for notic-ing, or longer term effects+ Even though the second study tried to lessen theuse of memorization strategies and even though, as stated previously, short-term awareness, language use, or restructuring might be considered evidenceof acquisition ~Norris & Ortega, 2003!, the results must still be interpretedcautiously because the learners did not have to transfer their knowledge to anovel situation over a long period of time+ To make truly fair and pedagogi-cally useful comparisons of these feedback methods, researchers should lookat the long-term outcomes associated with employing them repeatedly; thisstudy should in no way be seen as support for the claim that written errorcorrection works in the long term+

Moreover, and not too surprisingly, when verbalization data provided accessto at least some of what the participants were noticing in the reformulation �think-aloud condition, we found that changes in accuracy on revisions andverbalized instances of awareness did not always correspond+ That is, the learn-ers occasionally did not incorporate corrections into their revisions even whenthey had shown themselves to understand them in their verbalizations, andthey occasionally corrected linguistic items that they had not mentioned ~anobservation also made by Qi & Lapkin, 2001!+Whether the experience of notic-ing might be helpful for learners in processing relevant features of languagein future input or output is a question for further research+


What this observation also means is that our online measures of attentionmight have suffered from nonveridicality in addition to reactivity; it is notentirely clear what the participants were aware of in the reformulation � think-aloud condition, nor can the learners’ verbalizations be interpreted as straight-forwardly illustrating the unreported attentional processes in the reformulationcondition+ Correspondences between a reformulation and revision sometimesmade it clear that the former had influenced the latter ~e+g+, if a learner wrotelish, trying to use the new word leash that he or she had presumably seen inthe reformulation!+ However, at other times it was much more difficult to tell~e+g+, if a learner changed on to in after possibly noticing at in the reformulation!+

One of our original interests in designing this study was to see if it wouldbe possible to quantitatively confirm Qi and Lapkin’s ~2001! assertion that thequality of noticing experienced by a learner while processing feedback couldhave direct implications for revisions of L2 writing+ According to Qi and Lap-kin, substantive noticing ~i+e+, noticing with a reason or, perhaps analogously,awareness at the level of understanding! might have more of an impact onlearning than perfunctory noticing ~i+e+, noticing without providing a reason!+We, on the other hand, do not intend to claim a cause-effect relationshipbetween noticing and subsequent linguistic accuracy based on the results ofour study, as Qi and Lapkin might have implied in theirs+ It is not possible forus to declare that the fact that participants verbalized certain things—or theway in which they did so—actually caused errors to be changed in the revi-sions+ Our data suggest associations; for instance, errors that were noticedwere more likely to be changed, and when the learners used metalanguage orgave reasons for errors, the errors concerned were also more likely to bechanged than not+ However, this might relate to other factors, including thelearners’ prior knowledge of the forms and their readiness or ability to noticeand talk about them+

Furthermore, although Qi and Lapkin’s ~2001! distinction between substan-tive and perfunctory noticing seems straightforward enough on the surface~either a learner gives a reason, or he or she does not!, the actual coding ofthe learners’ verbalizations for evidence of various sorts of awareness turnedout, in practice, to be a rather high-inference process+ For instance, a learnerwho recognized an accidental oversight ~coded O in Tier II of our system!mightappear to know a reason on some level even without stating it explicitly—forexample, “Right! They were worried! @laughs# Why I put worry? Yeah, right!Worried+” In this case, the participant does not provide an explicit reason orany overt evidence of understanding why worried is a better choice than worry;however, the noticing seems to go beyond being perfunctory+

Another possibility to consider is that the type of error in question mightinfluence what is verbalized+ In this study, feedback did not focus exclusivelyon a particular set of target forms, and the learners’ verbalizations made itclear that they were successful in noticing a wide range of error types+ How-ever, pinning down the phenomenon of awareness in metalinguistic verbaliza-tions with regard to such a wide variety of error types can be problematicfrom a methodological standpoint+ In noticing a mistake in subject-verb agree-


ment or spelling, for instance, recognition might be quick enough comparedto the rest of the verbalization process that learners feel it unnecessary toverbalize ~or simply do not verbalize! an underlying reason for the error anddo not engage in any further analysis of it+ A clear-cut case of perfunctorynoticing would seem to be merely reading a correction aloud without com-menting on it at all ~a coding of RD in our system!+ Mentioning a correctionand repeating it with emphasis ~a coding of M! might also seem perfunctory+However, here it becomes difficult to draw the line+ For example, what is thedifference between simply mentioning a mistake ~e+g+, “Oh, it’s look at” or “Oh,I need at”!, on the one hand, and using metalanguage ~e+g+, “Oh, I need a prep-osition”!, on the other?

Thus, to approach coding learners’ verbalizations from the point of view oftrying to distinguish between perfunctory and substantive noticing, some fac-tors that might interact with a researcher’s ability to do so include the com-plexity of the error, the difficulty involved in understanding and correcting it,and the facility with which it lends itself to verbalization ~related to the ques-tion of veridicality in employing think-aloud protocols!+ Even the statement “Idon’t know” ~coded NR, lack of reason! might be substantive if arriving at thatconclusion involved a number of false starts and problems that could not beverbalized; moreover, there is a difference between not being able to identifythe target of a correction and not understanding the reason behind an identi-fiable correction+

Ultimately, then, as far as coding is concerned, substantiveness seems oftento be related to completeness of verbalization+ This might, in fact, be stronglyrelated to level of awareness and depth of processing, and this study showedit to be associated with revision changes+ Nonetheless, whereas an analysis ofwhat and how much is verbalized can enrich our understanding of awarenessand might be of pedagogical value, it does not appear to be possible to telldefinitively how deeply a correction has been processed and why it laterappears in a revision based solely on this approach+ An alternative approach,drawing from the attention and awareness strand of SLA research, might proveto be more effective at capturing the phenomena of interest+ Rather than com-paring varieties of perfunctory and substantive verbalizations, which differwith respect to whether a learner explicitly states a reason, it might be moreproductive to focus on whether the learner appears to experience subjectiveinsight ~i+e+, whether he or she is using language as language, on the one hand,or stepping back from it with some form of meta-awareness of language asobject, on the other!+ The difference is subtle, but the realities of coding learn-ers’ verbalizations would seem to argue for its legitimacy+

FUTURE RESEARCH

As discussed previously, Schmidt ~2001! proposed that whereas awareness atthe level of noticing consists of the perception of a linguistic form, awarenessat the level of understanding involves noticing plus metalinguistic awareness


or metacognition ~i+e+, thinking about thinking, analyzing, comparing, experi-encing conscious insight, or attempting to understand the significance of alinguistic form!+ Leow’s ~1997! coding system for awareness is useful in thisregard, in that it involves determining whether a learner has ~a! demonstrateda behavioral or cognitive change ~e+g+, by producing a given form verbally orin writing!, ~b! demonstrated being subjectively aware of the experience, or~c! described a metalinguistic rule+ He takes the first to represent awarenessat the level of noticing, whereas the second two represent the level of under-standing+ Bringing this approach into future studies of L2 feedback process-ing would likely provide insights into the relationships between learners’awareness of language and their subsequent use of it; it could also serve tocheck the applicability of Leow’s coding method—employed thus far to inves-tigate learner awareness on L2 problem-solving and reading tasks—in the areaof L2 writing+

At the same time, it is important to reiterate that whereas level of aware-ness might very well be influential or facilitative in some way, the revisionsthat learners make are not necessarily due strictly to their noticing of exter-nal feedback+ Learners might demonstrate higher levels of awareness whenthey are ready to learn and use the structures that later show up in their revi-sions, when they have recently been taught certain grammar rules, or simplywhen they have certain strategies or orientations toward feedback+ In our study,participants sometimes corrected linguistic items on their revisions even whenthey had apparently not noticed them during the comparison stage; at othertimes, they did not correct the mistakes that they had mentioned+ In oral inter-action research,Mackey and Philp ~1998! pointed out that the immediate uptaketo a recast might be a red herring—in other words, not necessarily predictiveof whether the learner will subsequently make use of it+ In their study, regard-less of immediate uptake, those learners who were developmentally ready toacquire question forms ~in terms of Pienemann and Johnston’s, 1987, hierar-chy! were more likely to do so ~see Mackey, 1999, and Philp, 2003, for analo-gous findings!+ A fruitful area for future research might be the influence ofvarious sorts of prior knowledge and individual differences on learners’ notic-ing and uses of feedback+ Different types of familiar and novel linguistic itemsshould be investigated as well, especially considering Mackey et al+’s ~2000!finding that feedback is more or less likely to be recognized as such, depend-ing on the area of language to which it pertains+

Strictly speaking, because exactly the same types of corrections were madein all of the conditions ~except for the control!, relative visual salience wasthe main difference between the error corrections and reformulations thatwere given to the students+ The experimental conditions of this study ~whichinvolved, e+g+, imposing time limits, eschewing prewriting activities, not tar-geting specific features of language for feedback, and taking the correctionsaway from the students! were not meant to exemplify good practice in L2writing pedagogy, and no recommendations are made regarding their use inthe classroom+ However, the results seem to indicate that the role of percep-tual salience in feedback processing merits further investigation+ Robb et al+


~1986! described the four methods of providing feedback in their study asdiffering in terms of salience and did not find differences among them in termsof long-term effectiveness, as noted previously; however, it is important tonote that the types of feedback they compared ~i+e+, complete correction, error-type coding, and the marking of precise and general error locations! pro-vided varying amounts of information to the learners+ Because these methodscomprise various degrees of directness as well as salience, the extent to whichtheir results relate to salience per se is questionable+ In the context of thecurrent study, we might have been able to gain useful information regardingsalience through a qualitative examination of differences between learners’verbalizations in error correction � think-aloud and reformulation � think-aloud conditions+ More generally, further research that isolates salience ~orperhaps certain aspects of salience! as an independent variable would behelpful+

Time concerns provide another area worthy of examination+ Among theexperimental conditions of this study, the error correction condition appar-ently led to a more efficient use of time in terms of the particular analyseswe conducted+ What remains uninvestigated, however, is what would havehappened had the learners been given more than 15 min to process the feed-back they received+ Making reference to Robinson ~1995!, Qi and Lapkin ~2001!stated that “even noticing with comprehension may need some reinforcedrehearsal in memory” ~p+ 295!, and this might have been something the errorcorrection condition provided when the feedback was salient and the learn-ers had time with it+ In the current study, some of the learners mentionedhaving been able to make use of memorization, error classification, and count-ing strategies in the error correction condition, but not in the other condi-tions+ It might be worthwhile to investigate whether an initial stage ofcommitting feedback to memory and rehearsing it is helpful beyond a singlerevision+ Citing research on the learning of both vocabulary ~Baddeley,Papagno, & Vallar, 1988! and morphosyntax ~N+ Ellis, 1996! in relation to learn-ers’ working memory capacities, Philp ~2003! emphasized that rehearsal allowslearners to consolidate the L2 input they encounter, make comparisonsbetween IL and TL, and process this information sufficiently for long-termstorage+ If various aspects of language can be acquired through implicit analy-ses of memorized chunks, as N+ Ellis ~1996, 2005! has claimed, then rehearsalin working memory might be particularly important+ Research should alsoinvestigate the long-term effects of encouraging learners to process teach-ers’ corrections and reformulations in order to explore relationships amongwritten feedback, awareness, and longer term acquisition+

~Received 18 July 2006!

NOTES

1+ Truscott ~2004! refuted the findings from Chandler’s ~2003! experimental study that showedthe effectiveness of error correction, and we agree that her results might be problematic+


2+ Although this study was very well designed, reliability on the accuracy measures was notfully reported; thus the results should be interpreted cautiously+

3+ Because Qi and Lapkin ~2001! investigated differences between levels of awareness, their resultsshould not be construed as directly relevant to Schmidt’s noticing hypothesis, which claims onlythat awareness at the level of noticing is necessary for language learning, not that awareness at thehigher level of understanding is necessary+ By extension, in the current study, we will differentiatebetween awareness at the level of noticing and awareness at the level of understanding, not address-ing whether noticing is necessary for SLA+

4+We decided to have the participants perform their think-alouds in English primarily so thatthe researchers could understand them+ However, because this was the learners’ L2, these think-alouds should not be confused with the method of pure concurrent verbalization described byEricsson and Simon ~1993!+ The participants might have experienced additional demands on mem-ory and attentional resources, which could have affected not only their thought processes butalso the ideas that they were able to express ~see, e+g+, Mackey et al+, 2000, who found that theaverage number of words per stimulated recall comment was 26 for a group of learners speakingin their L1 but only 16 for learners speaking in their L2!+ One should perhaps also consider, how-ever, that using a L1 to discuss aspects of a L2 might interfere with language processing in a dif-ferent way+

5+ For example, if a learner originally used an incorrect preposition and then changed that prep-osition to another incorrect preposition in the revision, this was coded as a change+

6+ Given that each error had already been coded individually, it might seem like an unfortunateloss of information to have compared percentages at the level of T-units, thereby ignoring howmany and what types of errors were changed in each T-unit+ However, the decision to do so wasprincipled+ Looking over the data, it became evident that comparing the conditions with respect tothe percentages or types of errors changed might be misleading+ First, some students introducednew, unrelated errors of a certain type and then repeated them throughout the revisions, making asimple comparison of error quantity from story to revision impractical and unrepresentative+ Forinstance, one participant introduced 400% more article errors just by making one mistake ~unrelatedto any of the corrections that had been made! and repeating it several times over the course of hisrevision+ Similarly, a student might be able to notice one overarching problem ~e+g+, verb tense inrelation to text cohesion! and then correct all of the related errors at once+ Analyzing changes inaccuracy at the level of T-units did not completely circumvent this problem, but it addressed manyaspects of it+

REFERENCES

Baddeley, A+, Papagno, C+, & Vallar, G+ ~1988!+ When long-term learning depends on short-term stor-age+ Journal of Memory and Language, 27, 586–595+

Bowles, M+ A+, & Leow, R+ P+ ~2005!+ Reactivity and type of verbal report in SLA research methodol-ogy: Expanding the scope of investigation+ Studies in Second Language Acquisition, 27, 415–440+

Braidi, S+ ~2002!+ Reexamining the role of recasts in native-speaker0non-native speaker interactions+Language Learning, 52, 1–42+

Carroll, S+ E+ ~2001!+ Input and evidence: The raw material of second language acquisition+ Amsterdam:Benjamins+

Chandler, J+ ~2003!+ The efficacy of various kinds of error feedback for improvement in the accuracyand fluency of L2 student writing+ Journal of Second Language Writing, 12, 267–296+

Choi, M+-Y+ ~2000!+ Effects of recasts on irregular past tense verb morphology in web-chat+ Unpub-lished master’s thesis, University of Hawai‘i, Honolulu+

Cumming, A+ ~1990!+ Metalinguistic and ideational thinking in second language composing+ WrittenCommunication, 7, 482–511+

DeKeyser, R+ M+ ~1995!+ Learning second language grammar rules: An experiment with a miniaturelinguistic system+ Studies in Second Language Acquisition, 17, 379–410+

Egi, T+ ~2004!+ Recasts, perceptions and L2 development+ Unpublished doctoral dissertation, George-town University, Washington, DC+

Ellis, N+ C+ ~1996!+ Sequencing in SLA: Phonological memory, chunking, and points of order+ Studies inSecond Language Acquisition, 18, 91–126+

Ellis, N+ C+ ~2005!+ At the interface: Dynamic interactions of explicit and implicit language knowledge+Studies in Second Language Acquisition, 27, 305–352+

Ellis, R+ ~1994!+ A theory of instructed second language acquisition+ In N+ C+ Ellis ~Ed+!, Implicit andexplicit learning of languages ~pp+ 79–114!+ San Diego: Academic Press+


Ellis, R+ ~2001!+ Introduction: Investigating form-focused instruction+ Language Learning, 51~Suppl+ 1!,1–46+

Ericsson, K+ A+, & Simon, H+ A+ ~1993!+ Protocol analysis: Verbal reports as data ~Rev+ ed+!+ Cambridge,MA: MIT Press+

Ferris, D+ ~1999!+ The case for grammar correction in L2 writing classes: A response to Truscott~1996!+ Journal of Second Language Writing, 8, 1–11+

Ferris, D+ ~2004!+ The “grammar correction” debate in L2 writing: Where are we, and where do we gofrom here? ~and what do we do in the meantime+ + +?!+ Journal of Second Language Writing, 13,49–62+

Fuchs, M+, Fletcher, M+, & Birt, D+ ~1986!+ Around the world: Pictures for practice. Book 2+ London:Longman+

Gass, S+ M+ ~1997!+ Input, interaction and the second language learner+ Mahwah, NJ: Erlbaum+Iwashita, N+ ~2003!+ Positive and negative input in task-based interaction: Differential effects on L2

development+ Studies in Second Language Acquisition, 25, 1–36+Jourdenais, R+ ~2001!+ Cognition, instruction and protocol analysis+ In P+ Robinson ~Ed+!, Cognition

and second language instruction ~pp+ 354–375!+ New York: Cambridge University Press+Kroll, B+ ~1990!+ What does time buy? ESL student performance on home versus class compositions+

In B+ Kroll ~Ed+!, Second language writing: Research insights for the classroom ~pp+ 140–154!+ NewYork: Cambridge University Press+

Laufer, B+, & Hulstijn, J+ ~2001!+ Incidental vocabulary acquisition in a second language: The con-struct of task-induced involvement+ Applied Linguistics, 22, 1–26+

Leeman, J+ ~2003!+ Recasts and second language development: Beyond negative evidence+ Studies inSecond Language Acquisition, 25, 37–63+

Leow, R+ P+ ~1997!+ Attention, awareness, and foreign language behavior+ Language Learning, 47, 467–506+Leow, R+ P+ ~1999!+ The role of attention in second0foreign language classroom research: Methodolog-

ical issues+ In J+ Gutierrez-Rexach & F+ Martinez-Gil ~Eds+!, Advances in Hispanic Linguistics ~pp+ 60–71!+ Somerville, MA: Cascadilla Press+

Leow, R+ P+ ~2001!+ Do learners notice enhanced forms while interacting with the L2? An online andoffline study of the role of written input enhancement in L2 reading+ Hispania, 84, 496–509+

Leow, R+ P+, & Morgan-Short, K+ ~2004!+ To think aloud or not to think aloud: The issue of reactivity inSLA research methodology+ Studies in Second Language Acquisition, 26, 35–57+

Loewen, S+, & Philp, J+ ~2006!+ Recasts in the adult L2 classroom: Characteristics, explicitness andeffectiveness+ Modern Language Journal, 90, 536–556+

Long, M+ H+ ~1996!+ The role of the linguistic environment in second language acquisition+ In W+ C+Ritchie & T+ K+ Bhatia ~Eds+!, Handbook of language acquisition: Vol. 2. Second language acquisi-tion ~pp+ 413–468!+ San Diego: Academic Press+

Long, M+ H+ ~2007!+ Problems in SLA+ Mahwah, NJ: Erlbaum+Lyster, R+ ~1998!+ Recasts, repetition, and ambiguity in L2 classroom discourse+ Studies in Second

Language Acquisition, 20, 51–81+Lyster, R+ ~2004!+ Differential effects of prompts and recasts in form-focused instruction+ Studies in

Second Language Acquisition, 26, 399–432+Lyster, R+, & Ranta, L+ ~1997!+ Corrective feedback and learner uptake+ Studies in Second Language

Acquisition, 19, 36–66+Mackey, A+ ~1999!+ Input, interaction and second language development: An empirical study of ques-

tion formation in ESL+ Studies in Second Language Acquisition, 21, 557–587+Mackey, A+, Gass, S+ M+, & McDonough, K+ ~2000!+ How do learners perceive interactional feedback?

Studies in Second Language Acquisition, 22, 471–497+Mackey, A+, & Philp, J+ ~1998!+ Conversational interaction and second language development: Recasts,

responses, and red herrings? Modern Language Journal, 82, 338–356+Makino, T+ ~1993!+ Learner self-correction in EFL written compositions+ ELT Journal, 47, 337–341+Nicholas, H+, Lightbown, P+ M+, & Spada, N+ ~2001!+ Recasts as feedback to language learners+ Lan-

guage Learning, 51, 719–758+Norris, J+ M+, & Ortega, L+ ~2000!+ Effectiveness of L2 instruction: A research synthesis and quantita-

tive meta-analysis+ Language Learning, 50, 417–528+Norris, J+ M+, & Ortega, L+ ~2003!+ Defining and measuring L2 acquisition+ In C+ J+ Doughty & M+ H+

Long ~Eds+!, The handbook of second language acquisition ~pp+ 717–761!+ Oxford: Blackwell+Philp, J+ ~2003!+ Constraints on “noticing the gap”: Nonnative speakers’ noticing of recasts in NS-NNS

interaction+ Studies in Second Language Acquisition, 25, 99–126+Pienemann, M+, & Johnston, M+ ~1987!+ Factors influencing the development of language proficiency+

In D+ Nunan ~Ed+!, Applying second language acquisition research ~pp+ 45–141!+ Adelaide, Austra-lia: National Curriculum Resource Centre, AMEP+


Polio, C+ ~1997!+ Measures of linguistic accuracy in second language writing research+ Language Learn-ing, 47, 101–143+

Polio, C+, Fleck, C+, & Leder, N+ ~1998!+ “If only I had more time”: ESL learners’ changes in linguisticaccuracy on essay revisions+ Journal of Second Language Writing, 7, 43–68+

Polio, C+, & Gass, S+ M+ ~1998!+ The role of interaction in native speaker comprehension of nonnativespeaker speech+ Modern Language Journal, 82, 308–319+

Qi, D+ S+, & Lapkin, S+ ~2001!+ Exploring the role of noticing in a three-stage second language writingtask+ Journal of Second Language Writing, 10, 277–303+

Robb, T+, Ross, S+, & Shortreed, I+ ~1986!+ Salience of feedback on error and its effect on EFL writingquality+ TESOL Quarterly, 20, 83–95+

Roberts, M+ ~1995!+ Awareness and the efficacy of error correction+ In R+ Schmidt ~Ed+!, Attention andawareness in foreign language learning ~pp+ 163–182!+ Honolulu: University of Hawai‘i Press+

Robinson, P+ ~1995!+ Attention, memory, and the “noticing” hypothesis+ Language Learning, 45,283–331+

Robinson, P+ ~2001!+ Individual differences, cognitive abilities, aptitude complexes and learning con-ditions in second language acquisition+ Second Language Research, 17, 368–392+

Rosa, E+ M+, & Leow, R+ P+ ~2004!+ Awareness, different learning conditions, and second language devel-opment+ Applied Psycholinguistics, 25, 269–292+

Rosa, E+ M+, & O’Neill, M+ ~1999!+ Explicitness, intake, and the issue of awareness: Another piece ofthe puzzle+ Studies in Second Language Acquisition, 21, 511–556+

Sachs, R+, & Suh, B+-R+ ~in press!+ Textually enhanced recasts, learner awareness, and L2 outcomes insynchronous computer-mediated interaction+ In A+ Mackey ~Ed+!, Conversational interaction insecond language acquisition: A series of empirical studies+ Oxford: Oxford University Press+

Saxton, M+ ~1997!+ The contrast theory of negative input+ Journal of Child Language, 24, 139–161+Schmidt, R+ W+ ~1990!+ The role of consciousness in second language learning+ Applied Linguistics, 11,

129–158+Schmidt, R+ ~1995!+ Consciousness and foreign language learning: A tutorial on the role of attention

and awareness in learning+ In R+ Schmidt ~Ed+!, Attention and awareness in foreign language learn-ing+ Honolulu: University of Hawai‘i Press+

Schmidt, R+ ~2001!+ Attention+ In P+ Robinson ~Ed+!, Cognition and second language instruction~pp+ 3–32!+ New York: Cambridge University Press+

Schmidt, R+, & Frota, S+ ~1986!+ Developing basic conversational ability in a second language: A casestudy of an adult learner of Portuguese+ In R+ Day ~Ed+!, Talking to learn: Conversation in secondlanguage acquisition ~pp+ 237–326!+ Rowley, MA: Newbury House+

Sheen, Y+ H+ ~2004!+ Corrective feedback and learner uptake in communicative classrooms acrossinstructional settings+ Language Teaching Research, 8, 263–300+

Stratman, J+ F+, & Hamp-Lyons, L+ ~1994!+ Reactivity in concurrent think-aloud protocols: Issues forresearch+ In P+ Smagorinsky ~Ed+!, Speaking about writing: Reflections on research methodology~pp+ 89–112!+ Thousand Oaks, CA: Sage+

Swain, M+ ~1985!+ Communicative competence: Some roles of comprehensible input and comprehen-sible output in its development+ In S+ M+ Gass & C+ Madden ~Eds+!, Input and second languageacquisition ~pp+ 235–256!+ Rowley, MA: Newbury House+

Swain, M+ ~1995!+ Three functions of output in second language learning+ In G+ Cook & B+ Seidlhofer~Eds+!, Principles and practice in applied linguistics ~pp+ 125–144!+ Oxford: Oxford University Press+

Swain, M+ ~1998!+ Focus on form through conscious reflection+ In C+ J+ Doughty & J+ Williams ~Eds+!,Focus on form in classroom second language acquisition ~pp+ 64–81!+ New York: Cambridge Uni-versity Press+

Swain, M+, & Lapkin, S+ ~1995!+ Problems in output and the cognitive processes they generate: A steptowards second language learning+ Applied Linguistics, 16, 371–391+

Thornbury, S+ ~1997!+ Reformulation and reconstruction: Tasks that promote “noticing+” ELT Journal,51, 326–335+

Tomlin, R+ S+, & Villa, V+ ~1994!+ Attention in cognitive science and second language acquisition+ Stud-ies in Second Language Acquisition, 16, 183–203+

Truscott, J+ ~1996!+ The case against grammar correction in L2 writing classes+ Language Learning,46, 327–369+

Truscott, J+ ~2004!+ Evidence and conjecture on the effects of correction: A response to Chandler+Journal of Second Language Writing, 13, 337–343+

VanPatten, B+ ~1990!+ Attending to form and content in the input+ Studies in Second Language Acqui-sition, 12, 287–301+

Wong, W+ ~2001!+ Modality and attention to meaning and form in the input+ Studies in Second Lan-guage Acquisition, 23, 345–368+


APPENDIX A

EXAMPLE OF A WRITING PROMPT

Note+ All three picture sequences used in this study were adapted from Aroundthe World: Pictures for Practice, Book 2, by M+ Fuchs, M+ Fletcher, and D+ Birt,1986, White Plains, NY: Longman, and were used with permission+ Picturedhere is The Dinner Party ~pp+ 42–43!+ The other two sequences, not pictured,are Jogging ~pp+ 30–31! and Bank Robbers ~pp+ 14–15!+


APPENDIX B

EXAMPLE OF COLUMNS FORMAT: STUDY 2, STUDENT 13 IN THE REFORMULATION + THINK-ALOUDCONDITION

Table B1. Story and reformulation

Story Reformulation

One day, he noticed that his tammy is kind of terrible by looking atthe mirror+

One day, while looking in the mirror, a man noticed that his tummylooked pretty terrible+

Near by the mirror, there was a book titled “Get in Shape”+ Nearby the mirror, there was a book entitled “Get in Shape+”He decided to start jogging+ He decided to start jogging+He looked he was filled with a bunch of enagy+ He looked as though he was filled with a bunch of energy+He read the book to know “how to jog”+ He read the book in order to find out how to jog+As he was jogging, his tammy was shaked, As he was jogging, his tummy was shaking,and his way of jogging was kind of strange+ and his jogging style was kind of strange+Everyone was pointing out him and laughing+ Everyone was pointing at him and laughing+He was embarrassed+ He was embarrassed+As soon as he turened the corner, he found two wealthy females+ As soon as he turned the corner, he found two wealthy females+One of them had a dog by holding a rope+ One of them was holding a dog on a leash+

96R

ebeccaS

achsand

Charlene

Polio

Table B2. Think-aloud and revision

Think-aloud Revision

OK, um, uh, I wrote, first of all, I wrote, he noticed that his tummy iskind of terrible, but native speaker’s one is first of all, while looking inthe mirror, a man noticed that his tummy looked pretty terrible, terri-ble+ Mmm+ + + I don’t know why+ I think + + + first of all, when I wrotethis, I thought this is, I tried to write sentence + + + correctly, so I don’tknow why this is, why they, there is difference+Hm+ By looking at the mirror, and while looking at the mirror, whilelooking+ Ah, and I also didn’t know that when I used, when somebodyuses the word while, I thought a person has to put sub-subject andverb and + + + but this time, she doesn’t use any subject between whileand looking+ So + + + that’s my, that’s what I notice+

One day, while he was watching the mirror, he noticed that his tammywas kind of terrible+

Mmmm + + + I wrote there was a book titled “Get in Shape,” but anotherone’s + + + there was a book entitled “Get in Shape+” Hm+ Maybe Ishould have wrote, written, entitled+

Near by the mirror, there was a book antitled “Get in Shape+”He decided to start jogging+

He looked he was filled with a bunch of energy+ He looked he wasfilled+ + + He was, he looked he was filled with a bunch of energy+That’s what I wrote, and he looked as though he was filled with abunch of energy + + + mmm+ I didn’t write “as though+” Hm+ Maybe if Iwrote “as though” it’s more, much more very, very more clear+

*He looked he was filled with a bunch of enagy+

He read the book to know how to jog+ He read the book in order tofind out how to jog+ He read the book to know how to jog+ Hm+ Inorder to find, find out+ It’s + + + makes more sense+

He read the book in order to know “how to jog+”

As he was jogging, his tummy was shaked, shaking+ Hm+ As he was jogging, his tammy was shaking,

Note+ The learners did not receive their reformulations in this format+ They received full-page copies of their original stories, not separated into T-units+ This columns format aided the researchersin coding and is presented here for ease of comparison+

Learners’U

sesof

Written

Feedback

97

APPENDIX CComparison of conditions with regard to changes in accuracy for individualparticipants ~expressed as percentages!

Condition

Participant Error correction Reformulation Think-aloud

A 93+8 83+3 92+9B 100+0 100+0 92+3C 100+0 93+8 78+3D 100+0 100+0 81+8E 100+0 91+7 100+0F 93+3 93+3 70+6G 94+4 85+0 91+7H 100+0 95+2 100+0I 94+4 88+9 82+6J 69+2 53+9 64+3K 100+0 92+9 66+7L 100+0 100+0 79+0M 100+0 86+7 71+4N 100+0 100+0 92+9O 100+0 84+6 56+5Total ~N � 15! 96+4 90+0 81+4

APPENDIX DTable D1. Comparison of conditions with regard to complete correctionsof T-units for individual participants ~expressed as percentages!

Condition

Participant Error correction Reformulation Think-aloud

A 43+8 16+7 14+3B 25+0 20+0 23+1C 14+3 12+5 8+7D 90+0 83+3 45+5E 41+7 25+0 40+0F 53+3 66+7 29+4G 22+2 10+0 25+0H 69+2 52+4 10+0I 11+1 27+8 17+4J 30+8 15+4 7+1K 73+3 35+7 44+4L 83+3 30+0 26+3M 43+8 20+0 4+8N 81+3 47+4 28+6O 22+2 15+4 8+7Total ~N � 15! 47+0 31+9 22+2


Table D2. Friedman test of ranked percentages

Mean rank of each condition Test statistics

Error correction Reformulation Think-aloud N x2 df p

2+73 1+87 1+40 15 13+733 2 +001*

*p , +05 ~statistically significant!+

APPENDIX E

QUESTIONS AND COMMENTS FROM POSTSTUDY DEBRIEFING

Questions

1+Which activity was the easiest for you to do?2+Which was the most difficult?3+ Did one of the activities make it easier to remember the corrections?4+ Did you use any strategies when you were comparing?5+ Do you think your strategies changed over time?6+Which activity did you like best?7+Which one do you think was the most useful?

Participant Comments

Participant J: I think correction is easiest, very, more easy than others because, in speech, whenyou speak for the first time, try to, it is very difficult to me because I have to speakwhy I, my sentence is wrong, so + + + I, I, I can understand why I + + + sentence is wrong,but I can’t speak well, yeah, so it is very difficult, and this also, compare, compari-son activity also, I have to do searching why, what I + + + yeah, search, and so some, ifI missed some words, I can, if I wrote some, missed some word, I, uh, just keep + + +uh, and correction activity, you write down, so, oh!, so I can search more easier, so Ican’t + + + at, when I wrong word, if I, when I write down wrong word, oh! I can find it,so + + + yes+

Participant F: I think correction activity is more understand+ Easier, more easier+ Because morefamiliar, I think, more familiar, when I watched this paper, I feel it’s more familiar+And + + + when I watched this paper, I felt, I recognized, this is wrong and this is right+I felt like that, so I think correction activity is more easy+

Participant F: The most difficult thing is comparison because I, actually, I, I, um, that is very diffi-cult to me, the compare, compare about, um, my case and the reviser case+ It isdifficult to distinguish+ It is so difficult to distinguish+

Participant G: Reformulation and think-aloud is both are important, are very useful for me+ Because+ + + the reformulation I have to very, look carefully+ I have to look carefully the arti-cle+ Then I have to find the difference+ It’s very interest- useful, useful, helpful+ Thethink-aloud is very also useful because I have to think again, then, at that time, firstwriting the essay, sometimes I write down even when I don’t know, I’m not sure thegrammar or spelling or + + + but I + + + I have to think again about the paper, so it’s


very useful+ Then I can think again, so I think it’s very useful+ The correction is notuseful for me because you already corrected + + + you used a different color pen orsomething, so I can find very easy to other mistake, ah, no, the difference+ So + + + Ijust look faster to find color, Oh! I found it! Because it’s very easy to see a differ-ence between my paper and ~the one! you gave me+ The paper makes it very easyto find the difference, so I don’t have to concentrate the paper+

Participant M: Talking is very hard, and+ + + I don’t know grammar names+ For example, I can speaka relative clause, but I don’t know a lot of grammar mistakes+ So I can’t, I can’texplain my mistake+

Participant I: If I say something, my thinking is very fast because first I think and then I speak+But reading is, uh, I, I can think, think enough time+ Yeah, and + + + um, and, yeah, Ican think enough time, and I can think deeply+ And sometimes I can memorize mymistakes, so last time I, I rewrite correct+ But the think-aloud is, uh, actually, myproblem is I, I forget everything easily+ That is my problem+ So I think this classactivity is, uh, easy to memorize+


Sachs_Polio_2007.pdf

Documents

study learners

language learners

learners grammatical

adult learners of english

reactivity of think

error correction condition

department of linguistics

georgetown edu