RESEARCH POSTER PRESENTATION DESIGN © 2012 www.PosterPresentations.com This study compared the use of cohesion devices and lexical knowledge in L1 and L2 blog entries, using the Coh-Metrix tool (http://tool.cohmetrix.com). The present study is based on the research described in Crossley and McNamara (2009) who studied the lexical differences in L1 and L2 writing by using Coh-Metrix. The topic of automated measures of L2 writing has attracted considerable interests recently, due to potentially better efficiencies and higher effectiveness of computational measures, compared to human raters. In the present study, the Coh-Metrix tool (http://tool.cohmetrix.com) is used to automatically evaluate lexical knowledge and the use of cohesive devices in the L2 (English) writing of an Indonesian writer, and to compare these measures with those generated for L1 writers. In my study, written entries from an L2 writer’s (http://tantrialmira.wordpress.com/) and an L1 writer’s (http://jeremyharmer.wordpress.com/) blogs are analysed for indices of lexical knowledge and cohesion, and the results are compared. Introduction Research Question: Design & Methodology Results Implications Limitations • A case study subjects of the study one L1 writer and one L2 writer • Sample texts: relatively low in number, raising the question of generalizability • Future studies using Coh-Metrix may benefit from using: a) large corpus b) not only cohesion and lexical indices References Crossley, S.A. & McNamara, D. S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18, 119–135 Crossley, S. A., & McNamara, D. S., (2010). Predicting second language writing proficiency: the roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35 (2), 1–21. DOI: 10.1111/j.1467-9817.2010.01449.x Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2010). Predicting lexical proficiency in language learner texts using computational indices. Language Testing, 20 (10), 1–20. doi:10.1177/0265532210378031 Graesser, A. C. & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3, 371-398 Kyle, K. (2011). Objective measures of writing quality. Master Thesis. Colorado State University, Fort Collins: USA McNamara, D. S., Crossley, S.A., & McCarthy, (2010). Linguistic features of writing quality. Written Communication, 27 (1), 57-86. doi: 10.1177/0741088309351547 McNamara, D. S., Louwerse, M. M., McCarthy, P.M., & Graesser, A. C. (2010). Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47 (4), 292-330 O’Reilly, T., & McNamara, D. S. (2007). Reversing the reverse cohesion effect: Good texts can be better for strategic, high-knowledge readers. Discourse Processes, 43 (2), 121-152 http://jeremyharmer.wordpress.com/ http://tantrialmira.wordpress.com/ Can indices of cohesion and lexical use generated by Coh-Metrix be used to distinguish between L1 and L2 blog writers? School of Linguistics and Applied Language Studies, Victoria University of Wellington Anik Wulyani, PhD candidate Automatic measures of cohesion and lexical proficiency in L2 writing: A case study Indices Expected Indices # 29: Argument overlap L1 > L2 Indices # 46: Latent Semantic Analysis (LSA) give/new L1 < L2 Indices # 85: Verb incidence L1 > L2 Indices # 94: CELEX word frequency for content words, mean L1 < L2 Indices # 97: Age of acquisition L1 < L2 Indices # 102: Polysemy L1 > L2 Indices # 103: Hypernymy for nouns L1 > L2 Indices Expected Indices # 48: Lexical diversity, Type-Token Ratio, content word lemmas L1 > L2 Indices # 49: Lexical Diversity, Type-Token Ratio, all words L1 > L2 Indices # 50: Lexical Diversity, MTLD, all words L1 > L2 Indices # 51: Lexical Diversity, VOCD, all words L1 > L2 Indices # 95: CELEX Log frequency for all words, mean L1 < L2 Indices # 99: Concreteness for content words L1<L2 Indices # 104: Hypernymy for verbs L1 > L2 Indices # 105: Hypernymy for noun and verbs L1 > L2 1. Practicality Coh-Metrix is able to distinguish L1 and L2 texts using linguistics features (cohesion and lexical indices) 2. Assessment Coh-Metrix is a powerful tool to measure L2 writing L 1 writer: 13 blog posts L2 writer: 13 blog posts Cohesion indices: 5 indices Lexical indices: 10 indices Cohesion indices: 5 indices Lexical indices: 10 indices Coh-Metrix Tool 3.0 compared MANOVA Are they significantly different? Can you see the difference? Automatic measures of cohesion and lexical proficiency in L2 writing: A case study Blogging among in-service teachers • Main research • EFL teachers in Indonesia • Pilot study • Coh-Metrix Tool 3.0 • One L1 blog writer, native speaker of English, the UK • One L2 blog writer, EFL teacher, Indonesia Those 15 indices from Coh-Metrix were chosen because previous related studies showed that they were able to differentiate L1 writing to L2 writing (Crossley & McNamara, 2009; Crossley & McNamara, 2010; Kyle, 2011; McNamara, Crossley & McCarthy, 2010; & Crossley, Salsbury, McNamara, & Jarvis, 2010). Today I guide my students to continue design their summative task and formative tasks. I ask my students to list down possible things they can do for their summative and formative tasks. I asked them to list down as many action as possible (actions which they have done in their previous summative or their lies). Then they need to classify the actions which can be feasibly done for their summative tasks. In my blog I have tried, so far, to address general issues to do with presenting, conferences, writing abstracts etc etc. As with every other blogger, my ‘thoughts’ have been personal, of course, but I have tried to exercise some dispassion. But not this time. Just for once I want to tell you how I feel – or rather what it felt like (and then see if there is anything to learn from that). What I am trying to say is that this post is going to be incredibly personal, and I hope you will forgive me for that. Sample 1 Blog post Sample 2 Blog post Future Research Future research English Language Teaching in second language context L2 writing proficiency L2 blog writers cohesion and lexical indices Before discussing the findings of the study, please check the samples of the blog posts I have put in the next section! In terms of cohesion and vocabulary use which one do you think is the blog post of an L1 writer? How do you know? What does Coh-Metrix 3.0 and MANOVA with alpha level .05 tell us about the cohesion and lexical use? Current study versus Crossley & McNamara (2009) Indices Current study Crossley & McNamara (2009) Argument overlap, #29 L1 < L2 L1 > L2 LSA given/new, #46 L1 < L2 L1 < L2 Verb Incidence, #85 L1 < L2 L1 > L2 Lexical diversity , #50 L1 > L2 L1 > L2 Lexical diversity, #51 L1 > L2 L1 > L2 Hypernymy for nouns & verbs, #105 L1> L2 L1 > L2 Crossley & McNamara (2009) Present study Seven variables (indices) Fifteen variables (indices) Corpus of L1 (undergraduates in USA) & of L2 (the International Corpus of Learner English) Blog posts of L1 (a teacher and trainer in English Language Teaching) & of L2 (an Indonesian English teacher) Academic writing/argumentative essays Blog writing Table 2b Expected Results, Additional Indices Table 1 Differences between Crossley & McNamara’s study & present study Table 2a Expected Results (Crossley & McNamara, 2009) 1. Three cohesion indices showed significant differences: argument overlap (#29), LSA given/new (#46), and verb incidence (#85) 2. Three lexical indices reached statistical differences: lexical diversity #50, #51, and hypernymy for nouns and verbs (#105) 3. Cohesion: a) The L1 writer appeared to produce more cohesive texts than the L2 writer b) The L2 writer was more likely to produce texts with more new information than the L1 writer. c) The L2 writer seemed to use more verbs or more spatial cohesion than the L1 writer 4. Lexical Proficiency: a) The L1 writer seemed to have more diverse lexicon than the L2 writer b) The L2 writer appeared to use less diverse vocabulary than the L1 writer c) The L1 writer used significantly more conceptually abstract and hierarchically connected words than the L2 writer Why are the cohesion indices of the present study different from Crossley & McNamara (2009)? Because: a) The L1 writer appeared to be more casual in organizing his texts (few argument overlap) and to expect his readers to have high knowledge to understand his texts (few verb incidences) b) The L2 writer seemed to have a formal writing or text organization (greater argument overlap) and to use more cohesion devices (more verb incidences) to assist her readers to better understand her texts Graesser and McNamara, 2011; McNamara, Louwerse, McCarthy, & Graesser, 2010; & O’Reilly & McNamara, 2007 Email address: [email protected]