THE CONTEXT OF THE PROJECT – Language Testing in Russia Carolyn Westbrook Southampton Solent University PROSET: A step forward in testing and assessment.

THE CONTEXT OF THE PROJECT – Language Testing in Russia

Carolyn Westbrook

Southampton Solent University

PROSET: A step forward in testing and assessment for Russian teachers of English Library of Foreign Literature, Moscow 17 September 2013

Outline 1. Construct validity - what are we assessing?

2. Test validation - quality control in language assessment

3. The context of the ProSET project

4. Key project milestones

a. Language testing and assessment training – Luton, UK

b. Development of modules

c. Piloting

d. Monitoring visits

Construct validity – what are we assessing?

German listening test - what are we testing here?

Listen to two people talking about their pets and match the words from the text to the definitions:

1. Katze a. dog2. Hund b. mouse3. Kaninchen c. cat4. Maus d. rabbit

Construct validity – what are we assessing?

English reading test - what are we testing here?

Read the text and answer the questions below in full sentences using your own words.

1. According to the report, what other role have public relations firms taken on?

Construct validity – what are we assessing?

The moral of the story …• Think very carefully about the

purpose of your test: • What do you want to find out?• Which skills do you want to test?• How are you going to test them?• Are the tasks going to provide the

information you need?

Test validation – quality control in language assessment

Validation aims to achieve “explanation and understanding. Therefore, the profession is coming around to the view that all validation is construct validation”. (Cronbach, 1984: 126, in Messick, 1989: 19).

Test validation – quality control in language assessment

Threats to construct validity: 1. Construct underrepresentation: the

test “fails to include important dimensions or facets of the construct”

2. Construct irrelevance:the test is irrelevantly difficult or easy.

(Messick, 1989: 34)

Test validation – quality control in language assessment

Test design: test specifications

How will it be scored?

Who?

How many?

Why?

What?

How long?

How will it be answered?

Test taker characteristics

Test purpose / abilities / characteristics

Test tasks / types of items

Number of items and tasks

Length of test

Response type

Scoring criteria

Context of the project

• High quality language teaching• Background in language assessment• Move to EGE • Need for understanding of theoretical

principles of testing

Project milestones2012 2013

TASK F M A M J J A S O N D J F M A M J J A S O N DLTA training - Luton                                              ECTS training - Dresden                                              Development of modules                                              1st round monitoring                                              Revision of modules                                              2nd round monitoring                                              Final revision of modules                                              Handover conference                                              Publishing of materials                                              

The future for language testing in Russia

• Language testing and assessment and ECTS modules published

• Regional PDCs to train both pre- and in-service teachers

• for long-term sustainability • Improved knowledge and understanding of

theoretical language testing

Any questions or comments?

[email protected]