ACTFL WPT Review_Yinghua Cai

  1

Unveiling the ACTFL Writing Proficiency Test (WPT)

Yinghua Cai

EDUC8540: Language Assessment

Dr. Kathleen Bailey

October 29, 2012

  2

Introduction

As an MA TESOL candidate aspiring to teach not only English but also Mandarin

Chinese after graduation, I have decided to review the language-neutral ACTFL Writing

Proficiency Tests, which are now available in 18 languages, including Arabic, English,

Chinese, French, German, Hebrew, Hindi, Italian, Japanese, Korean, Persian Farsi, Polish,

Portuguese, Russian, Serbian-Croatian, Spanish, Urdu, and Vietnamese.

Bailey (1998) suggests that “the main purpose of language assessment” is “to help

us gain the information we need about our students’ abilities and to do so in a manner

that is appropriate, consistent, and conducive to learning.” (p. 2) A parallel argument can

be made that the main purpose of a test review is to help us gain the information we need

about a test and to use the test in a manner that is appropriate, consistent, and conducive

to learning.

I am hoping that this test review can be informative and beneficial to students

who are going to take the ACTFL Writing Proficiency Test (WPT) as well as other tests

that are developed on the basis of the ACTFL Proficiency Guidelines 2012 – Writing.

Moreover, this review aims at informing the language teachers who are going to use the

ACTFL WPT for decision-making related to curriculum design and development as well

as student admission and placement.

This test review consists of four major sections: the history of the ACTFL

Proficiency Guidelines and the ACTFL WPT, the description of the ACTFL WPT, the

analysis of the ACTFL WPT using Wesche (1983) and Swain’s (1984) frameworks, and

the examination of the scoring system.

History of the ACTFL Proficiency Guidelines and the ACTFL WPT

  3

According to Swender (2012), the ACTFL Proficiency Guidelines globally

characterize what users of a language can and cannot do with regard to a hierarchy of

global tasks. As time goes on, the hierarchical structure has been expanding with major

levels and sublevels added and specified. Informed by the ILR Language Skill

Descriptions, the first version of the ACTFL Proficiency Guidelines was published in

1986 and became widely used since then (ACTFL, 1986). The 0 to 5 scale in the ILR

descriptors were changed to Novice (0), Intermediate (1), Advanced (2), Superior (3, 4,

5). Meanwhile, Novice and Intermediate levels were divided into Low, Mid, and High

sublevels. In 2001, Advanced Low and Advanced Mid Sublevels were added to the

Guidelines for Writing (ACTFL, 2001). And it was the 2001 revision of Guidelines for

Writing that coincided with the development, validation, and operationalization of the

ACTFL WPT (ACTFL, 2002).

With the increased use of the Guidelines, a new round of revision started in 2009

to address the currency and relevancy of the Guidelines and to revisit the critical borders

(e.g., Advanced High/Baseline Superior, Intermediate High/Advanced Low) for Speaking

and Writing (Swender, 2012). Three years later, the ACTFL published its newest

Proficiency Guidelines in a downloadable pdf format (see Appendix A for level

descriptions for Writing). A major level above Superior (Distinguished) for both

Speaking and Writing, a general level description (Novice, Intermediate, and Advanced),

and a Glossary of Terms have been added; guidelines with level-specific exemplars (in

English) for all skills also become available via the ACTFL website.

Swender (2012) argues that adding Distinguished to Speaking and Writing can be

attributed to practical considerations for testing. Actually, some test takers are able to test

  4

beyond the Superior level. With the notion of the Distinguished level in mind, students

may target higher linguistic proficiency as their educational goals and continue to

enhance their language proficiency in the post-education professional world.

Description of the ACTFL WPT

The ACTFL Writing Proficiency Test Familiarization Manual defines the ACTFL

WPT as “a standardized test for global assessment of functional writing ability in a

language” (p. 4). As a direct test (Bailey, 1998), the ACTFL WPT taps into a person’s

ability to compose effective and appropriate samples for real-life writing purposes rather

than the enabling skills, such as locating errors in an existing writing sample, therefore

the elicited written responses deal with a variety of practical, social, and professional

topics that are encountered in informal and formal real-life contexts (ACTFL, 2012).

The ACTFL WPT is also an integrative test. It “addresses a number of abilities

simultaneously and looks at them from a global perspective rather than from the point of

view of the presence or absence of any given linguistic feature”; “linguistic components

are viewed from the wider perspective of how they contribute to the overall writing

sample” (ACTFL, 2012, p. 5). As a criterion-referenced test (Brown, 2005), the ACTFL

WPT “measures how well a person spontaneously writes in a language (without access to

revisions and/or editing tools) by comparing his/her performance of specific writing tasks

with the criteria stated in the ACTFL Proficiency Guidelines 2012 – Writing” and “does

not compare one writing sample to another” (ACTFL, 2012, p. 4).

As for its purpose, the ACTFL WPT is a proficiency test that is “independent of

any particular curriculum” and provides “a general assessment of all prior learning and/or

acquisition” (Bailey, p. 40). As a common metric for describing a person’s functional

  5

writing ability, the ACTFL WPT provides parity among language programs. It can

inform admission, placement, and fulfillment of exit/graduation decisions for language

programs. It can also be used to evaluate the effectiveness of language programs.

Moreover, the ACTFL Proficiency Guidelines – Writing may serve as a framework for

designing and developing multi-level writing curriculum. Last but not least, The ACTFL

WPT ratings are not only used for teacher certification but also for hiring and promotion

in multi-lingual positions in the business world and in government (ACTFL, 2012).

Currently, the ACTFL WPT is available in booklet form or online with adaptive

prompts. Individual and group tests can be scheduled by contacting:

Language Testing International (LTI)

ACTFL Testing Office

[email protected]

Tel: (914) 963-7110

Analysis of the ACTFL WPT

By using the Wesche’s (1983) Framework, I have summarized the key

components of the ACTFL WPT (online version) in the below table and will elaborate on

each component in the following paragraphs based on my experience taking the online

demo version for Spanish.

Stimulus material

Background Survey, Self-Assessment, Introduction and Warm-up

(about 10 minutes), and four separate requests (prompts) for a

variety of writing tasks in English

Task posed to the

learner

Understanding four requests (normally in English, unless special

accommodations requested) and completing written responses to

  6

the specific requests in the target language within the allotted 80

minutes

Learner’s response

A test taker’s written responses (e.g., several paragraphs)

Scoring criteria

Holistic rating scale derived from The ACTFL Proficiency

Guidelines 2012 – Writing; encompassing four major levels,

namely Superior, Advanced, Intermediate, and Novice

Stimulus material refers to both linguistic and nonlinguistic information presented

to the learners to get them to demonstrate the skills or knowledge to be assessed (Bailey,

1998). The ACTFL WPT system first requires test takers to finish a Background Survey

(Appendix B) that asks about their work, school, home, personal activities, and interests

as well as a Self-Assessment (Appendix C) that provides six different descriptions of how

well a person can write in a target language. Based the information gathered from the

Background Survey and Self-Assessment, a customized test is generated to get test takers

to demonstrate their writing proficiency.

Following the Background Survey and the Self-Assessment is the Introduction

and Warm-up section, which contains directions, information on test-taking strategies,

and a warm-up activity at the Novice Level, such as filling in personal information

(Spanish demo test). About 10 minutes are allotted for this section of the test. Then come

the four separate writing prompts (see Appendix D for sample writing prompts), each of

which describes the audience, context, and purpose of the prompt as well as the suggested

  7

length of the response and time allotment for completing the response to a specific

request.

The task posed to the learner and the learner’s response “go hand-in-hand”

(Bailey, 1998, p. 13). It is quite obvious that the task posed to the learner is to complete

written responses to specific prompts in the target language based on the understanding

of the prompts. Considering that all the prompts are written in English unless special

accommodations are requested, the test validity may suffer if a test taker, who is a non-

native English speaker, starts responding in a target language without fully understanding

the English prompts. I am concerned that the ACTFL WPT might end up measuring the

test taker’s English reading proficiency rather than the target language writing

proficiency, thus deviating from the purpose of the ACTFL WPT (Brown, 2005).

According to Writing Proficiency Test Familiarization Manual 2012, the ACTFL

Proficiency Guidelines 2012 – Writing is the basis for scoring the ACTFL WPT,

encompassing four major levels spanning a full range of writing abilities, from Superior

to Novice (see Appendix A for level descriptions). When evaluating the writing sample,

the following aspects are considered, namely “the functions or global tasks the writer

performs”, “the social contexts and specific content areas within which the writer

performs the tasks”, “the accuracy of the writing”, and “the length and organization of the

written discourse the writer produces” (ACTFL, 2012, p. 5).

Swain’s (1984) Framework can also be applied to analyzing the ACTFL WPT and

will bring in some new perspectives. How the ACTFL WPT fits into Swain’s (1984) four

principles of communicative language test design is summarized in the below table and

illustrated in detail in the following paragraphs.

  8

Start from somewhere

Preliminary proficiency guidelines – Writing revised 2001

and evolving guidelines

Concentrate on content

Customized test; real-life writing tasks; ACTFL Proficiency

Guidelines revised, addressing current topics and tasks

Bias for best

Keyboard options; stimulus material; Writing Proficiency Test

Familiarization Manual 2012 (including ACTFL Proficiency

Guidelines 2012 – Writing); English writing samples and

glossary of terms; specific annotations for Arabic and

language-specific samples; previous versions of ACTFL

Proficiency Guidelines

Work for washback

ACTFL Proficiency Guidelines – Writing serving as a step

ladder for language learning

The ACTFL Proficiency Guidelines – Writing is the starting point for the ACTFL

WPT. Breiner-Sanders, Swender and Terry (2002) published revised guidelines for

writing proficiency in languages and these guidelines were used to create the earliest

version of the ACTFL WPT. With the publication of the new Guidelines (ACTFL, 2012),

the WPT is expected to evolve and develop accordingly.

In terms of concentrate on content, the ACTFL WPT offers customized tests that

take into account test takers’ background (e.g., interests, self-perceived proficiency

level). The traditional paper and pencil format has the same customization and adaptive

features as the online version that addresses individualized content. Additionally, the

writing tasks posed can be very meaningful and relevant to the test taker. For example,

  9

citing an experience that prepares one for a new position can be part of the cover letter

that a test taker needs to prepare in order to get a job interview (Appendix D).

Since the ACTFL Guidelines has been revised to reflect communication in the

21st century by describing “instant messaging, e-mail communication, texting” (ACTFL,

2012, p. 10) as interpersonal written text in the Preface, I believe that such revision may

also be shown in the evolving ACTFL WPT to include most up-to-date topics and tasks

(content) as the WPT has been developing out of the ACTFL Guidelines.

Bailey (1998) suggests that tests be designed to “elicit the best possible

performance from the test-takers” (p. 154). The WPT is designed in a way that makes the

test-taking process comfortable and convenient, which is conducive to desirable

performance. For example, for test takers to make diacritical marks in the target language

unrepresented on a standard US Keyboard, several keyboard options are available

(ACTFL, 2012). Moreover, presenting instructions on how to navigate the test, relevant

test-taking strategies, and a warm-up activity also help set the momentum for the actual

test performance, potentially contributing to bias for best. The easily accessible Writing

Proficiency Test Familiarization Manual 2012, online English writing samples and

glossary of terms (context-specific to the Guidelines), Guidelines with specific

annotations for Arabic and language-specific samples, and the previous versions of the

ACTFL Proficiency Guidelines may also lead to best possible test performance if test

takers have a clear understanding of what they should be aiming for as they compose

their writing sample in the target language.

The ACTFL Proficiency Guidelines – Writing, out of which the rating scale is

developed, works for washback as well. It is noteworthy that the Guidelines not only

  10

defines what a test taker is able to accomplish but also what he/she is not, thus giving the

test taker a direction for further improvement. Teachers can also adjust the writing

instruction to deal with students’ noticeable weaknesses based on their performance on

the WPT.

Examination of the Scoring System

As described in the scoring criteria section, a global rating to a sample is assigned

based on the ACTFL Proficiency Guidelines – Writing. ACTFL (2012) describes the

following scoring procedure. When placing a writing sample within a major range and

then matched to the sublevel (see Appendix A for level descriptions), the ACTFL

certified rater also establishes patterns of strength and weakness identified in the writing

sample to justify the assigned rating. Under the supervision of Language Testing

International (LTI), the ACTFL Testing Office, each Official WPT is blindly double

rated by two separate ACTFL certified WPT raters, who have completed a rigorous

training and demonstrated ability to consistently rate samples with a high degree of

reliability.

At first glance, the scoring system seems legitimate. However, it does not address

how to deal with divergent ratings given by the two certified raters nor does it tackle the

unexpectedly conflicting performances of a single test taker. For example, what if one

rater assigns a writing sample Advanced Mid, but the other rater assigns the same sample

Intermediate High? Do we average the ratings and award Advanced Low? Or do we find

a third rater to rescore the writing sample? If a test taker demonstrates very high

proficiency in finishing the most complex writing task but performs badly in the other

three tasks, how should a rater assign a global rating to the writing sample? These

  11

concerns do not seem to have been addressed in any of the literature available to the

public so far.

Although the LTI’s quality assurance program is said to consistently monitor and

review WPT ratings (LTI, 2012), LTI has not published any public-accessible researches

regarding the validity and reliability of the official ACTFL WPT. Surface, Dierdorff and

Poncheri (2006) claim that their research “represents the first assessment of reliability

and validity evidence for the WPT” (p. 2). In their study, a total of 509 Writing

Proficiency Tests were conducted and rated by experienced certified testers complying

with the ACTFL WPT assessment procedure. Inter-rater reliability for the full sample

(n=509) was .94 and for a Spanish-only sample (n=395) was .92. This study also provides

evidence of validity by relating WPT results to OPI results for a subgroup (n=460). It is

found that the OPI and WPT assess overlapping constructs (r= .81; p< .001) regardless of

the skill modality difference. Since both speaking and writing are productive skills, one

might expect a significant relationship.

Given the limited empirical data on test reliability and validity, further research is

needed to examine and document those psychometric properties of the WPT, which is in

line with the Standards for Educational and Psychological Testing (1999). More

importantly, research on test reliability and validity may inform potential revision of the

current ACTFL WPT, thus establishing a more accurate measurement tool.

Conclusion

The ACTFL WPT is not as robust as the ACTFL Oral Proficiency Interview

(OPI), with rather limited print/online resources available to the public. The language

assessment professionals and writing instructors I am familiar with at the Monterey

  12

Institute of International Studies also do not have first-hand experience taking or

administering the WPT. Moreover, my experience with the Spanish demo version is

rather inauthentic since only two sample prompts are generated no matter how I complete

the Background Survey and Self-Assessment. All these reflect the less-than-developed

and less-than-prominent status of the current ACTFL WPT.

However, I still believe that the ACTFL WPT, grounded in well-articulated

Proficiency Guidelines, can be a promising assessment tool that serves various learner

populations and purposes. Therefore, I am looking forward to becoming a certified

ACTFL WPT rater and hopefully conducting another review of the ACTFL WPT in

several years.

  13

References

American Council on the Teaching of Foreign Languages. (2012). ACTFL writing

proficiency test familiarization manual. White Plains, NY: Author.

American Council on the Teaching of Foreign Language. (2002). ACTFL writing

proficiency test familiarization guide. Yonkers, NY: Author.

American Council on the Teaching of Foreign languages. (2001). ACTFL proficiency

guidelines. Hastings-on-Hudson, NY: Author.

American Council on the Teaching of Foreign Languages. (1986). ACTFL proficiency

guidelines. Yonkers, NY: Author.

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (1999). Standards for

educational and psychological testing. Washington, DC: Author.

Bailey, K. M. (1998). Learning about language assessment: Dilemmas, decisions and

directions. Boston: Heinle & Heinle.

Breiner-Sanders, K. E., Swender, E., & Terry, R. M. (2002). Preliminary proficiency

guidelines – Writing revised 2001. Foreign Language Annals, 35(1), 9–15.

Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English

language assessment. New York: McGraw-Hill.

Language Testing International (2012). Writing proficiency test. Retrieved from

http://www.languagetesting.com/writing-proficiency-test

Surface, E. A., Dierdorff, E. C., & Poncheri, R. M. (2006, May). The “write” stuff: A

preliminary assessment of a writing proficiency test. Paper presented at the 21st

  14

annual conference of the Society for Industrial and Organizational Psychology,

Dallas, TX.

Swain, M. (1984). Large-scale communicative language testing: A case study. In S. J.

Savignon, & M. Berns (Eds.), Initiatives in communicative language teaching

(pp. 185-201). Reading, MA: Addison-Wesley.

Swender, E. (2012). Introducing the ACTFL proficiency guidelines 2012 [pdf document].

Retrieved from

http://www.govtilr.org/Publications/ACTFL%20Guidelines%202012%20ILR%20

Presentation%201.6.pdf

Wesche, M. B. (1983). Communicative testing in a second language. The Modern

Language Journal, 67, 41-55.

  15

Appendix A

Level Descriptions

DISTINGUISHED Writers at the Distinguished level can carry out formal writing tasks such as

official correspondence, position papers, and journal articles. They can write analytically on professional, academic and societal issues. In addition, Distinguished-level writers are able to address world issues in a highly conceptualized fashion.

These writers can use persuasive and hypothetical discourse as representational techniques, allowing them to advocate a position that is not necessarily their own. They are also able to communicate subtlety and nuance. Distinguished-level writing is sophisticated and is directed to sophisticated readers. Writers at this level write to their audience; they tailor their language to their readers.

Distinguished-level writing is dense and complex; yet, it is characterized by an economy of expression. The writing is skillfully crafted and is organized in a way that reflects target-culture thought patterns. At the Distinguished level, length is not a determining factor. Distinguished-level texts can be as short as a poem or as long as a treatise.

Writers at the Distinguished level demonstrate control of complex lexical, grammatical, syntactic, and stylistic features of the language. Discourse structure and punctuation are used strategically, not only to organize meaning but also to enhance it. Conventions are generally appropriate to the text modality and the target culture. SUPERIOR

Writers at the Superior level are able to produce most kinds of formal and informal correspondence, in-depth summaries, reports, and research papers on a variety of social, academic, and professional topics. Their treatment of these issues moves beyond the concrete to the abstract.

Writers at the Superior level demonstrate the ability to explain complex matters, and to present and support opinions by developing cogent arguments and hypotheses. Their treatment of the topic is enhanced by the effective use of structure, lexicon, and writing protocols. They organize and prioritize ideas to convey to the reader what is significant. The relationship among ideas is consistently clear, due to organizational and developmental principles (e.g., cause and effect, comparison, chronology). These writers are capable of extended treatment of a topic which typically requires at least a series of paragraphs, but can extend to a number of pages.

Writers at the Superior level demonstrate a high degree of control of grammar and syntax, of both general and specialized/professional vocabulary, of spelling or symbol production, of cohesive devices, and of punctuation. Their vocabulary is precise and varied. Writers at this level direct their writing to their audiences; their writing fluency eases the reader’s task.

Writers at the Superior level do not typically control target-language cultural, organizational, or stylistic patterns. At the Superior level, writers demonstrate no pattern of error; however, occasional errors may occur, particularly in low-frequency structures. When present, these errors do not interfere with comprehension, and they rarely distract the native reader.

  16

ADVANCED Writers at the Advanced level are characterized by the ability to write routine

informal and some formal correspondence, as well as narratives, descriptions, and summaries of a factual nature. They can narrate and describe in the major time frames of past, present, and future, using paraphrasing and elaboration to provide clarity. Advanced-level writers produce connected discourse of paragraph length and structure. At this level, writers show good control of the most frequently used structures and generic vocabulary, allowing them to be understood by those unaccustomed to the writing of non-natives. Advanced High

Writers at the Advanced High sublevel are able to write about a variety of topics with significant precision and detail. They can handle informal and formal correspondence according to appropriate conventions. They can write summaries and reports of a factual nature. They can also write extensively about topics relating to particular interests and special areas of competence, although their writing tends to emphasize the concrete aspects of such topics. Advanced High writers can narrate and describe in the major time frames, with solid control of aspect. In addition, they are able to demonstrate the ability to handle writing tasks associated with the Superior level, such as developing arguments and constructing hypotheses, but are not able to do this all of the time; they cannot produce Superior-level writing consistently across a variety of topics treated abstractly or generally. They have good control of a range of grammatical structures and a fairly wide general vocabulary. When writing at the Advanced level, they often show remarkable ease of expression, but under the demands of Superior-level writing tasks, patterns of error appear. The linguistic limitations of Advanced High writing may occasionally distract the native reader from the message. Advanced Mid

Writers at the Advanced Mid sublevel are able to meet a range of work and/or academic writing needs. They demonstrate the ability to narrate and describe with detail in all major time frames with good control of aspect. They are able to write straightforward summaries on topics of general interest. Their writing exhibits a variety of cohesive devices in texts up to several paragraphs in length. There is good control of the most frequently used target-language syntactic structures and a range of general vocabulary. Most often, thoughts are expressed clearly and supported by some elaboration. This writing incorporates organizational features both of the target language and the writer’s first language and may at times resemble oral discourse. Writing at the Advanced Mid sublevel is understood readily by natives not used to the writing of non- natives. When called on to perform functions or to treat issues at the Superior level, Advanced Mid writers will manifest a decline in the quality and/or quantity of their writing. Advanced Low

Writers at the Advanced Low sublevel are able to meet basic work and/or academic writing needs. They demonstrate the ability to narrate and describe in major time frames with some control of aspect. They are able to compose simple summaries on familiar topics. Advanced Low writers are able to combine and link sentences into texts of paragraph length and structure. Their writing, while adequate to satisfy the criteria of the Advanced level, may not be substantive. Writers at the Advanced Low sublevel

  17

demonstrate the ability to incorporate a limited number of cohesive devices, and may resort to some redundancy and awkward repetition. They rely on patterns of oral discourse and the writing style of their first language. These writers demonstrate minimal control of common structures and vocabulary associated with the Advanced level. Their writing is understood by natives not accustomed to the writing of non-natives, although some additional effort may be required in the reading of the text. When attempting to perform functions at the Superior level, their writing will deteriorate significantly. INTERMEDIATE

Writers at the Intermediate level are characterized by the ability to meet practical writing needs, such as simple messages and letters, requests for information, and notes. In addition, they can ask and respond to simple questions in writing. These writers can create with the language and communicate simple facts and ideas in a series of loosely connected sentences on topics of personal interest and social needs. They write primarily in present time. At this level, writers use basic vocabulary and structures to express meaning that is comprehensible to those accustomed to the writing of non-natives. Intermediate High

Writers at the Intermediate High sublevel are able to meet all practical writing needs of the Intermediate level. Addition- ally, they can write compositions and simple summaries related to work and/or school experiences. They can narrate and describe in different time frames when writing about everyday events and situations. These narrations and descriptions are often but not always of paragraph length, and they typically contain some evidence of breakdown in one or more features of the Advanced level. For example, these writers may be inconsistent in the use of appropriate major time markers, resulting in a loss of clarity. The vocabulary, grammar, and style of Intermediate High writers essentially correspond to those of the spoken language. Intermediate High writing, even with numerous and perhaps significant errors, is generally comprehensible to natives not used to the writing of non-natives, but there are likely to be gaps in comprehension. Intermediate Mid

Writers at the Intermediate Mid sublevel are able to meet a number of practical writing needs. They can write short, simple communications, compositions, and requests for information in loosely connected texts about personal preferences, daily routines, common events, and other personal topics. Their writing is framed in present time but may contain references to other time frames. The writing style closely resembles oral discourse. Writers at the Intermediate Mid sublevel show evidence of control of basic sentence structure and verb forms. This writing is best defined as a collection of discrete sentences and/or questions loosely strung together. There is little evidence of deliberate organization. Intermediate Mid writers can be understood readily by natives used to the writing of non-natives. When Intermediate Mid writers attempt Advanced-level writing tasks, the quality and/or quantity of their writing declines and the message may be unclear. Intermediate Low

Writers at the Intermediate Low sublevel are able to meet some limited practical writing needs. They can create statements and formulate questions based on familiar material. Most sentences are recombinations of learned vocabulary and structures. These are short and simple conversational-style sentences with basic word order. They are

  18

written almost exclusively in present time. Writing tends to consist of a few simple sentences, often with repetitive structure. Topics are tied to highly predictable content areas and personal information. Vocabulary is adequate to express elementary needs. There may be basic errors in grammar, word choice, punctuation, spelling, and in the formation and use of non-alphabetic symbols. Their writing is understood by natives used to the writing of non-natives, although additional effort may be required. When Intermediate Low writers attempt to perform writing tasks at the Advanced level, their writing will deteriorate significantly and their message may be left incomplete. NOVICE

Writers at the Novice level are characterized by the ability to produce lists and notes, primarily by writing words and phrases. They can provide limited formulaic information on simple forms and documents. These writers can reproduce practiced material to convey the most simple messages. In addition, they can transcribe familiar words or phrases, copy letters of the alphabet or syllables of a syllabary, or reproduce basic characters with some accuracy. Novice High

Writers at the Novice High sublevel are able to meet limited basic practical writing needs using lists, short messages, postcards, and simple notes. They are able to express themselves within the context in which the language was learned, relying mainly on practiced material. Their writing is focused on common elements of daily life. Novice High writers are able to recombine learned vocabulary and structures to create simple sentences on very familiar topics, but are not able to sustain sentence-level writing all the time. Due to inadequate vocabulary and/or grammar, writing at this level may only partially communicate the intentions of the writer. Novice High writing is often comprehensible to natives used to the writing of non-natives, but gaps in comprehension may occur. Novice Mid

Writers at the Novice Mid sublevel can reproduce from memory a modest number of words and phrases in context. They can supply limited information on simple forms and documents, and other basic biographical information, such as names, numbers, and nationality. Novice Mid writers exhibit a high degree of accuracy when writing on well-practiced, familiar topics using limited formulaic language. With less familiar topics, there is a marked decrease in accuracy. Errors in spelling or in the representation of symbols may be frequent. There is little evidence of functional writing skills. At this level, the writing may be difficult to understand even by those accustomed to non-native writers. Novice Low

Writers at the Novice Low sublevel are able to copy or transcribe familiar words or phrases, form letters in an alphabetic system, and copy and produce isolated, basic strokes in languages that use syllabaries or characters. Given adequate time and familiar cues, they can reproduce from memory a very limited number of isolated words or familiar phrases, but errors are to be expected.

  19

Appendix B

Background Survey Sample Questions

  20

  21

  22

  23

Appendix C

Sample Self-Assessment

  24

Appendix D

Sample Writing Prompts

Only two prompts are available in the demo version.

Write a note to a friend who is visiting. Be sure to:

1. Greet him and say why you are writing.

2. Describe where you live.

3. List places you like to visit.

Be sure to write in Spanish.

Suggested length: 1 – 2 paragraphs

Suggested time: 10 minutes

You are filling out an employment application. The application form asks you to cite a

specific experience you have had that has prepared you for this new position.

1. Describe the experience in detail.

2. Explain what you learned through this experience.

3. State how or why this background will benefit a new employer.

Be sure to write in Spanish.

Suggested length: 2 – 3 paragraphs

Suggested time: 10 – 15 minutes

  25

Appendix E

English Writing Samples

Writing » Distinguished The following excerpt from the New York Times Book Review by Judith Shulevitz is an example of writing at the Distinguished level. Books

Books are sacred. We all know that, and lest we forget it, an entire literature exists to remind us. It includes essays by the greats – Montaigne, Benjamin, Borges, Roth – as well as books by the lesser known. But don’t be intimidated by the sheer size of the pro-book lobby, or the big names backing it. Try this as an exercise: suspend your bookist presumptions for a moment, then visualize the inside of your apartment or house.

It looks fine, right? More or less the way you’d like it to look, except for one thing. That is what to do with the books. They’ve piled up and piled up, and now they seem about to swallow you whole. Since I write about books, I am sent dozens every week. They invade in waves, like determined settlers. I fend them off. They arrive anyway. Their padded envelopes release balls of nonbiodegradable gray fluff that work their way into everything. Their publicity material falls out and floats out of reach. But that’s nothing compared with the stacks of books that take over the floor.

You may think that people who don’t write about books don’t have this problem, but if that’s what you think, you’re wrong. There are also many people who buy too many books. You’re probably one of them. You carry them home in spasms of hopefulness, then face the task of shelving them. But how? No solution you come up with ever quite works. One book-review editor I know just lets every corner of his house fill up with books; he owns so many that when he actually needs to consult one, he’ll buy a new copy rather that try to find the one he already owns. Other friends periodically purge themselves of their books, then find themselves checking out of the library a book they owned as recently as last month.

The problem isn’t The Book. It’s books. As a physical object, the book is a triumph of design: a rectangular volume of eye-catching matter that has evolved over millenniums to fit fairly neatly onto stackable shelves; a thing that is portable, handsome, mechanically reproducible and self-sufficient, and makes a pleasing rustle when in use. (Compare that with electronic books, which cause eyestrain, need batteries and emit an unpleasant whir.) As an emotional object, the book offers pleasure and enlightenment. Rationale for Rating This piece of Distinguished-Level writing is well structured and organized in a way that reflects target-culture thought patterns (e.g., question/answer format to imitate Socratic conversation–It looks fine, right? More or less the way you’d like it to look, except for one thing.). The writing is dense and complex while at the same time it shows an economy of expression (bookist presumptions , nonbiodegradable gray fluff ). The writer demonstrates control of complex lexical, grammatical, syntactic and stylistic features of the language (The problem isn’t The Book. It’s books.). While the Distinguished-Level writer always addresses sophisticated readers, there is also a conscious effort to write to one’s audience: a book lover writing to book lovers.

  26

Writing » Superior The following excerpt is an example of writing at the Superior level. Teachers

Many teachers have been complaining about our current student population. The implication is that they do not possess the same values as students of past generations. Some complain that the students do not appear to have the level of respect for those in authority that was once so evident in the schools and the community. Teachers are frustrated as students seem to place a higher value on play than their education.

Years ago this community was homogeneous. In most cases at least one parent or a grandparent stayed at home so that there was supervision of the students at all times. In today’s environment, the parents, largely new immigrants, find it necessary to work two jobs each and do not have the support network of close family members in the vicinity. The alienation that the newly arrived parents feel as they work in environments that are not pleasant combined with long hours causes them to arrive home feeling besieged on most days.

Under these circumstances the young parents are frustrated and confused as they listen to their children speaking the English that is taught to them in schools. Despite having brought their children to this country for a better life they wonder if so much sacrifice is worth what is happening to their families before their eyes. They silently wonder why the American dream seems so far away while they are working fifteen hour days, six days a week. In today’s diverse community everyone needs to understand the immigrants point of view as well as be sensitive to the cultural perceptions of other groups to reduce stereotyping and frustration. With this end in mind a regular meeting to discuss points of view and preconceived notions among community members with translators for each of the seven languages that are prevalent would no doubt add much to the understanding of our neighbors’ and teachers’ viewpoints.

Today’s teachers need to be culturally sensitive as well as supportive as students struggle to take their place in American society. While this newest wave of immigrants may seem so unusual, it needs to be remembered that just a generation or two ago our grandparents entered into this new world during a far simpler time. They entered a world where neighbors had the inclination and made the time to look after one another. A regularly scheduled discussion over time would serve to address people’s fears and preconceptions and allow for the building of understanding and compassion. Rationale for Rating

This piece of Superior-Level writing addresses a social topic–the plight of immigrant families–treated both abstractly and concretely. The writing uses a variety of sentence structures (They silently wonder why the American dream seems so far away ), syntax (Despite having brought their children to this country for a better life they wonder if so much sacrifice is worth what is happening to their families before their eyes.), and vocabulary (besieged). The writer demonstrates a strong awareness of writing for the audience (While this newest wave of immigrants may seem so unusual, it needs to be remembered that just a generation or two ago our grandparents entered into this new world during a far simpler time.). The relationship among ideas is consistently clear. This Superior-Level writer demonstrates a high degree of control of grammar, vocabulary—both general and specialized—and cohesive devices. As is typical of writers at the Superior Level, there is no pattern of errors in basic structures and the sporadic errors

  27

(understand the immigrants point of view) do not interfere with the communication or distract the reader.

Writing » Advanced The following is an example of writing at the Advanced Level. Alejandro Sanz

Yesterday, Tuesday, November 15th, 2005, the fans of the Spanish singer Alejandro Sanz, were able to see and hear his idol in person at the Radio City Music Hall in New York City. The tickets were sold out a month ago, and the performance of Alejandro Sanz was superior. As usual, Alejandro sang his most popular songs such as “El Alma al Aire” (“The Soul in the Air”), “Mi Soledad y Yo” (“My Lonelyness and I”), “Heroe” (“Heroe”), and more.

The concert started on time at 8pm and finished at 11pm. Three excellent hours of an outstanding performance, Sanz, gave to his fans from all over the world. The adrenaline of the fans was high with the romantic voice of the singer. Also, there were some lucky ones who were able to hold hands with their idol, and show their affection by giving to their favorite singer some gifts, such as flowers, teddy bears, chocolate boxes, etc.

Furthermore, the price of the tickets were from $85.00 to $300.00, and all the tickets were sold out a month ago. There is not doubt the Alejandro Sanz has won his fans’ heart. Personally, I believe that Alejandro Sanz is an excellent singer and his sense of humor, his personality, his passion for believing in the true love, makes him a unique singer. Rationale for Rating

This piece of Advanced-Level Writing reports on a topic of personal and general interest: a concert that was held the day before. The writer presents a complete narration/description in past time and includes elaboration to provide clarity (Also, there were some lucky ones who were able to hold hands with their idol, and show their affection by giving to their favorite singer some gifts, such as flowers, teddy bears, chocolate boxes, etc.). The writing is connected and cohesive in three logical paragraphs and shows good control of basic structures and vocabulary. This Advanced-Level writer expresses meaning that is comprehensible to those unaccustomed to non-native writing despite lexical, grammatical, and stylistic errors (Three excellent hours of an outstanding performance, Sanz, gave to his fans from all over the world.).

Writing » Intermediate The following is an example of writing at the Intermediate level. Email requesting a meeting

My name is Pascal. I am a member of the accounting department. I work here since 6 years. I am in charge of audit of your company. Nice to meet you. This e-mail is the first contact of us. I want to know each other. Please let me know when you can attend a meeting. How about 12 Feb or 14 Feb? And Where do you hope to take place? Please asertain the agenda of meeting. Regard. Pascal Rationale for Rating

This piece of Intermediate-Level writing is a simple e-mail message requesting a meeting and thus a basic exchange of information. It includes examples of

  28

straightforward questions (How about 12 Feb or 14 Feb? And Where do you hope to take place?). The topic is predictable and closely related to writer’s job of setting up a company audit. The writer produces loosely connected text composed primarily of discrete sentences (This e-mail is the first contact of us. I want to know each other.). The text requires some effort on the part of the reader who will need to be accustomed to the writing of non-natives (Please asertain the agenda of meeting.). Writing » Novice The following is an example of writing at the Novice level in response to the prompt “Leave a shopping list for your roommate.” School Supplies Pleez by 4 me– candy bannanas minerals watres printer papper sop toothpaste baterys Rationale for Rating

This example of Novice-Level writing is a list of basic words dealing with a most common aspect of a student’s daily life–shopping for essentials. The spelling may render part of this list difficult to comprehend even for readers who are accustomed to dealing with non-native writers (sop for soap).