Assessment Issues in Workplace ESL Instructional Programs

SymposiumWorkforce Assessment in the 21st Century

Knowledge Economy: A Quality Management Perspective

June 25, 2011

Assessment Issues in Workplace ESL Instructional Programs

MaryAnn Christison Dept. of Linguistics/Urban Institute for Teacher Education

University of UtahTIRF Board of Trustees

The International Research Foundation for English Language Education

Promoting research and best practices to improve the use of English in the 21st century global knowledge

economywww.tirfonline.org

A Backdrop for the Paper• Personal background – Instructional design, teacher education, and classroom-based

assessment and research– Experiences in workplace ESL include• identifying what type of language instruction is needed by a

given company (via a needs assessment),• designing language instruction with goals that meet

domain specific workplace needs, and• determining whether the language instruction is meeting

the goals.– Issues identified derive from practical experiences in

workplace language contexts

Five Issues

The issues • are not meant to comprise an exhaustive list,• are based on my own and others’ experiences, and• draw on data from my own research notes.

The Five Issues

1. Communicating to laypersons about the complex nature of language acquisition

2. Moderating learner and employer expectations about language

3. Connecting language training to workers’ on the job performances

4. Using assessment data in decision-making5. Understanding the research available on

workplace language instruction

Issue #1

• Communicating to laypersons about the complex nature of language acquisition

• Example Quotes“So, how long do you think it will take?”“Okay, so the good news is that I’ve been able to get funding for the next

six weeks [so that these employees to develop English language proficiency].”

“I’m really under the gun here. Of course, you know that I really support this program, but we just don’t understand why is it taking so long?”

“My son learned Italian in one summer, so I just don’t understand why can’t some of these people cannot speak English after two years?”

“I’m sorry to have to be the one to tell you this, but we have to cancel the program; it is too expensive and taking too long.”

Issue #1

• Difficulties in communicating to laypersons about the complex nature of language acquisition– Defining language constructs – Language vs. other competencies– Time and SLA– Time and money– Myths about language– Questions that we can answer about SLA are often not

questions laypersons are asking—there’s a cultural gap.

Issue #2

• The difficulty in moderating employer and employee expectations

• My first encounter• Diminishing U.S. federal funding for workplace literacy

in the mid to late 90s underscored the importance of accountability for workplace language educators. • Sponsoring companies and businesses had different

educational expectations for language programs than educational institutions and governmental agencies. Businesses and companies often compared language learning to the acquisition of non-language skills and wanted similar results relative to time.

Issue #2

• The difficulty in moderating employer and employee expectations

– Quotes from a hotel manager“I know . . ., but we had that a . . that a . . .consultant

come in last week for a workshop--six hours--, and now I think all of the employees can now use the software.”

“I think huh . . Mr. [X] still cannot enter this information and run this program, but that’s English.”

Issue #2• Companies are often not aware of their employees’ (and

potential employees’) expectations about assessment practices.– Quotes from potential flight attendants in an online chat room

“Does anyone know how they [will] evaluate us?” “Do you know if we will we have to read passenger instructions?” “Is there a spontaneous interview?” “Someone said there was a simulation or such and how can we prepare for that? Would they tell us, do you think?”“I’d like to talk to current crews, so I maybe can compare my language with theirs.”

Issue #2

– Companies can benefit from language assessment experts. Companies can improve skills in the administration of assessments and communicating about these assessments to potential employees.

Issue #3• Connecting language training to workers’

performances on the job– The beginnings • General English on-site • General Business English • Domain specific English (frequency vs. importance)

– Language tasks related to workplace tasks– Quotes from Call Centre online chat rooms

“The telephone role plays [were] very good for us.” “The interactive tasks are good I think, so we must talk to

others and make good impressions for our customers.”

Issue #4

– Using language assessment data in decision making.

Speaker A: “So, why don’t we talk about the results, OK? . . . Hm. . . I got your analysis yesterday. You know, a . . we might not be so happy with some things. Ah. . Hm. . Ok, so, for example, I think you know Mr. [X] and how good he is, but it seems that according to your, ya know test, hm . . . he didn’t do so well. OK. . . . a . . .So, it would be difficult to give him this hm . . promotion, and ah . . . Well, this needs to happen. I think we need to discuss a . . Hm. . . how to adapt these scores so that this happens. . . .”

Speaker B (consultant):“Ah. . . I don’t think I’m really clear on what you want me to do”

Issue #5

Understanding the research available on workplace instruction.

– Call Centre X reported a lack of success in using commercially available general business English tests within this industry. English tests provide a score for proficiency and do not break down the result in a diagnostic profile.

– Call centres reported that the general business tests were not valid indicators of what they were testing for. The scores did not carry meaning in terms of performance on the job.

– Neither the cost nor the long turn-around time met their needs.

Five Issues

– Communicating to laypersons about the complex nature of language acquisition

– Moderating learner and employer expectations about language

– Connecting language training to workers’ on the job performances

– Using assessment data in decision-making– Understanding the research available on

workplace language instructioncontact

[email protected]