ALTAANZ Conference 2016 In the classroom and beyond: Assessing language ability in different contexts November 17–19, 2016 The University of Auckland New Zealand
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ALTAANZConference2016Intheclassroomandbeyond:
Assessinglanguageabilityindifferentcontexts
November17–19,2016TheUniversityofAuckland
NewZealand
TABLEOFCONTENTS
1. MapShowingtheConferenceVenue 3
2. WelcomeMessagefromtheALTAANZPresidents 4
3. AboutALTAANZ 5
4. ALTAANZConferenceCommitteesandGroups 6
5. SponsorsandSupporters 8
6. ImportantInformation
9
7. Pre-ConferenceWorkshops 11
8. ConferenceDay1 13
9. ConferenceDay2 15
10. Teachers’Forum 18
11. Plenaries 19
12. Individualpaperabstracts 24
13. PapersinLanguageTestingandAssessment 41
14. PLTABestPaperAwards2013–2015 41
15. DELNA 42
16. PTEAcademic 43
17. IELTSResearchGrants 44
18. TOEFL 45
19. TheBritishCouncil 46
MAPSHOWINGTHECONFERENCEVENUE
Directions:TheConferencewillbeheldintheOwenGGlennBuilding(260)onGraftonRoad.Thisisthe
University’sBusinessSchool.YoucanaccesspaidparkingfromGraftonRoadasseenonthemap.Weekday
earlybird(entrybefore10.30am,exitby6.30pm)is$12;weekendflatrateis$6.
Foralargermap,pleasevisittheUniversityofAucklandwebsite,www.auckland.ac.nzandsearchformaps.
north
W
SE
00 2002005050 100100 150150
00 3311 22
SCALE 1:4 000 (1cm – 40m)SCALE 1:4 000 (1cm – 40m)
minutes (at 4km/h walking pace)minutes (at 4km/h walking pace)
metresmetres
CG0015
P
3
18
29
15B
15A
16A
5
7
7A
16
1V
4 4A
2
26
21
14
20A
20B
20
6
24
38
37
P
1
1616
1616
16
1616 1616
16
11
11
1
1
1
1
16
1
314314
109109
104104
106106
313313
315315
322322201201
206206
207207
219219
220220
250250240240
241241
810810
820820
801801
802802
803803
804804805805
135135
132132
123123
107107
122122
121121
120120
102102
110110
124124
125125
114114
113113103103
118118119119
273273
274274
275275226226
225225 253253
251251
252252
255255312312
260260
408408
404404
403403
402402
401401
423423
410410
421421
422422
409409 400400
200200212212
215215
216216
105105
800800
100 sector100 sector
311311
301301
303303
300300
813813 814814
637
638639
640
641
642
502
501
503
504505
630630
628
626
629
523 520526
518
522
528
529
532532
534534
599
614614
620620
600600
619619
439439
431431
432432
433433
440440
436436
435435400400
601601602602
603603
500 sector
600
901
902
903
904
907
900 sector
831831
832832
833833
834834
The StrandRailway Station(under construction)
911
908
910
302302
906
835835836836
837837
909
621
623633
632
ParnellRailway Station(under construction)
KK
JJ
II
HH
GG
FF
EE
DD
77 88 99 1010
1111 1212
KK
JJ
II
HH
GG
FF
EE
DD
88 99 1010 1111
1212
Newmarket Campus
NewmarketRailway Station
GraftonRailway Station
Mount EdenRailway Station
Grafton Campus
City CampusCity Campus
BritomartRailway Station
(underground)
AucklandFerry Terminal
Pier 1Pier 3-4
Pier 2
Staff ServiceStaff ServiceStudent ServiceStudent Service
Authorisedvehicles onlyAuthorised
vehicles only
RecreationCentre
RecreationCentre
GeneralLibraryGeneralLibrary
Old ChoralHall
Old ChoralHall
Biology
Biology
MaidmentMaidment
Stud
ent
Com
mon
s
Stud
ent
Com
mon
sKat
e Ed
ger
Info
Com
m
Kate
Edge
r
Info
Com
m
HumanSciencesHuman
Sciences
MusicMusic
Kenneth MyersCentre
Kenneth MyersCentre
NewmanHall
NewmanHall
FisherBuildingFisher
Building
MaclaurinChapelMaclaurinChapel
Old GovernmentHouse
Old GovernmentHouse
UniversityHouse
UniversityHouse
George FraserGallery
George FraserGallery
Comm
erce A
Comm
erce A
Elam
B
Elam
B Thom
as Bldg
Thom
as Bldg
Arts1
Arts1
Arts 2Arts 2
MaraeMarae
Fale PasifikaComplex
Fale PasifikaComplex
High CourtHigh Court
Pullman HotelPullman Hotel
CopthorneAnzac AveCopthorneAnzac Ave
UniLodgeUniLodge
Owen G Glenn BuildingOwen G Glenn Building
EngineeringEngineering
ConferenceCentre
ConferenceCentre
Alfred NathanHouse
Alfred NathanHouse
LodgeLodge
AucklandArt GalleryAuckland
Art Gallery
ScienceCentreScienceCentre
ScienceCentreScienceCentre
ParnellStudent Village
Grafton Hall
Starship Children’sHospital
Starship Children’sHospital
O’RorkeHall
O’RorkeHall
UniServicesHouse
UniServicesHouse
ElamSchoolElam
School
UniversityHall
UniversityHall
University HallApartments
University HallApartments
WhitakerHall
WhitakerHall
Auckland City HospitalAuckland City HospitalHuia
ResidenceHuia
Residence
Park RoadFlats
Main BuildingAuckland
War MemorialMuseum
Auckland
Domain
Auckland
Domain
Auckland
Domain
Auckland
Domain
The ClockTowerThe ClockTower
Albert
Park
Albert
Park
PORT OF AUCKLAND
MahuhukiterangiReserve
MahuhukiterangiReserve
Te Taou
Reserve
Te Taou
Reserve
Fraser ParkFrase
Scarborough
Reserve
Tahaki Reserve
WintergardenWintergarden
AucklandTown Hall
Aotea Centre
Myers
Park
Carlaw ParkStudent Village
Carlaw ParkStudent Village
Emily Place
Park
Emily Place
Park
Alten
Reserve
Alten
Reserve
Grafton
Cemetery
East
Grafton
Cemetery
East
Grandstand
Ferry Building
AucklandPublic Library
AucklandPublic Library
Newmarket Park
EndeavourPark
OlympicCorner
LumsdenGreen
StationSquare
OuthwaitePark
MountainReserve
Khyber PassReserve
BasquePark
Grafton Cemetery
West
Alberon
Reserve
BassettReserve
MonaMona
OkaretaOkareta
BelgraveBelgrave
Tennis
BayreuthBayreuth
ArchitectureArchitecture
John HoodPlaza
John HoodPlaza
CarltonPines
GraftonCampusGraftonCampus
88
5544
2277
55
3A3A
33
18181616
14141212
1010
66
33
2525
1515
111199
11
1616
1818
20-26
20-26
1616
1818
138
138
1616
181819
A19
A
25
25
7474
23
23
881
01
0
1133
5577
991212
1-111-11
1515
1717
99
22
22
2424
26
26
3434
3636
3838
2323
1717
1111
2020
2222
2424
2626
4040
8
133
133
1616
4949
7070
5858
14143030
2020
6767
2727
1616
9393
89
-91
89
-91
110
110
41 53
85
40
50
62
70
2-6
8
28
6
19-2
619
-26
38
10A10
Alfred
Street
Alfred
Street
Stre
et
Stre
et
AltenAlten
Ro
ad
Ro
ad
Anzac
Avenue
Anzac
Avenue
Waterloo
Quadrant
Waterloo
Quadrant
Parliament Street
Parliament StreetB
ow
enLa
ne
Bo
wen
Lan
e
BowenAvenue
BowenAvenue
Wyn
yard
Stre
et
Wyn
yard
Stre
et
Pri
nce
s
Stre
et
Pri
nce
s
Stre
et
CharlesN
aldenLn
CharlesN
aldenLn
Ch
urc
hill
Str
eet
Ch
urc
hill
Str
eet
Stre
et
Stre
et
Stan
ley
Stan
ley
Nicholls
Nicholls
Stan
ley
Stan
ley
Stre
etSt
reet
LowerDomain
LowerDomain DriveDrive
Car
law
Car
law
ParkPark
Avenu
e
Avenu
e
Gra
fton
Ro
ad
Gra
fton
Ro
ad
ShortStre
et
ShortStre
et
EdenEden
Crescent
Crescent
Pla
ceP
lace
Emily Place
Emily Place
EmilyEmily
Ba
nks
ide
Ba
nks
ide
Stre
etSt
reet
ShortlandShortlandStreetStreet
Stre
etSt
reet
Prin
ces
Prin
ces
Sym
onds
Sym
onds
ClaybrRoad
ook
Mount StreetMount Street
Saint
Saint Mart
ins
Lane
Mart
ins
Lane
Whitaker Place
Whitaker Place
Plac
ePl
ace
Wh
ita
ker
Wh
ita
ker
Sym
onds
Sym
onds
Stre
et
Stre
et
Park Road
Park
Ave
nu
e
Sea
fiel
d
V
iew
R
oa
d
Boyle Cres
Gla
sgo
wTe
rra
ce
Pa
rkR
oa
d
Pa
rkR
oa
d
Dom
ain
Grafto
nRoad
Grafto
nRoad
Gra
fton
Gra
fton
Ro
ad
Ro
ad Kari
Kari StreetStreet
Fencroft St
Fencroft St
CarltonGore
Road
CarltonGore
Road
Gra
fto
n
Road
BeckhamPlace
Wellesley
Street
Wellesley
Street
EastEast
St Paul StreetSt Paul Street
St Paul
Street
St Paul
Street
Wa
kefieldStreet
Wa
kefieldStreet
Lyndock St
Lyndock St
Aird
ale
Street
Aird
ale
Street
City
Street
Lane
Lane
Tenn
is
Tenn
is
GraftonBridge
GraftonBridge
gahape Road
Chancery StreetChancery Street
Ba
con
Lan
eB
aco
nLa
ne
Freyberg PlFreyberg Pl
Fie
lds
Fie
lds
Lan
eLa
ne
O’C
on
nel
l
Str
eet
O’C
on
nel
l
Str
eet
rthouseLane
rthouseLane
Co
uC
ou
Victoria Street E
Victoria Street E
Hig
hH
igh
Str
eet
Str
eet
Ha
lsey
Stre
et
Pakenham
Street East
Viaduct
Ha
rbo
ur
Avenue
Lucy
Lan
e
Ma
rket
P
lace
Ma
rket
La
ne
Sturdee Stre
et
Low
erH
ob
son
Stre
et
Cu
sto
ms
Stre
et
West
Customs
Street West
FanshaweStreet
Fanshawe
Street
QuayStreet
Customs Street East East
Qu
een
Str
eet
QuayStreet
Tyler Street
Tyler Street
Galway Street
Galway Street
Co
mm
erce
Co
mm
erce
StSt
Go
re S
tG
ore
St
Stre
et
Fort
Ln
Fort
Ln
Low
er
Alb
ert
Stre
et
Wolfe
Street
Fed
era
l
Str
eet
F
StreetStreet
SwansonSwanson
Mill
sM
ills
WyndhamStreet
Wyndham
Street
Wyndham
Street
Lan
eLa
ne
MillsLaneLane
Fed
era
l
Str
eet
KingstonStreet
Durham Lane
Durham Lane
DurhamStreet West
DurhamStreet West
Alb
ert
Stre
et
Alb
ert
Stre
et
Brandor Lane
Bo
uza
id W
ay
Gorst Lane
Vogel Lane
Graham
Stre
Ha
rdin
ge
Stre
et
et
Victoria Street West
Nel
son
Stre
et
Victoria
Street West West
Durham Street E
Durham Street E
VulcanLn
VulcanLn
Gore Street Lane
Gore Street Lane
FortFort StreetStreet
Brito
mar
t
Brito
ma
Pla
ce
Te Ara Tuhuhu
Walkway
Beach
Beach
Anza
cA
nza
c
Avenue
Avenue
StreetStreet
Kitch
Kitch
Kit
chen
er
S
tree
t
Kit
chen
er
S
tree
t
Lorn
eSt
reet
Lorn
eSt
reet
Lorn
eSt
reet
Lorn
eSt
reet
St PatricksSquare
Ho
bso
n
Stre
et
Qu
een
Str
eet
Qu
een
Str
eet
WellesleyStreet
West
Wellesley
Street West West
Sale St
Cook
StreetNel
son
Stre
et
ener
ener
Tin
ley
Stre
et
Plu
mer
S
tree
t
Tan
gih
ua
St
Tan
gih
ua
St
Te Taou Crescent
Te Taou Crescent
Te Taou CrescentTe Taou Crescent
CrescentCrescent
Cre
sC
res
Dockside Lane
Dockside Lane
RoadRoad
RoadRoad
Be
ach
Be
ach
RonayneRonayne
Street
Street
PlaceP
Ngahoe
Ngahoe
Ship
wri
ght
Ln
Ship
wri
ght
Ln
TheThe StrandStrand
The Strand
The
Str
an
d
QuayStreet
Gladstone Road
Faraday Street
Kenwyn Street
Wa
ttS
tree
t
Augustus
Terrace
EglonStre
et
Ma
rston
Street
Fox Street
York
Street
Earle Street
Earle
Street
Parnell R
oad
Tamaki Dr
Churton
Stre
et
Ba
th Street
Parnell Rise
Parnell
Bra
dford
Street
Street
Ba
th
GarfieldStreet
Garfield Street
Bra
dfo
rdStreet
Windsor
Street
Farnham Street
Cleveland Road
Sa
int
Geo
rges
Ba
yR
oa
d
Ruskin
Scarb
orough
StreetRuskin
Street
Terrace
Denby Street
Bedf
ord
Bedf
ord
Stre
et
Cra
crof
t S
tree
t
Mutu
StMutu
St
Stre
et
Stre
et
Heather Street
AkaroaStre
et
Tilden St
Falcon
Street
Tika St
Cheshire
Street
Gib
ralta
r
Crescen
t
Aorere Stre
et
Parn
ell Roa
d
Parnell Road
Scarborough Lane
Sa
int
Ge
org
es
Ba
yR
oa
d
Saint Stephens
Avenue
Birdw
ood
Crescent
Cathedral Place
Ayr
Street
Laurie
Avenue
Domain
Drive
Parn
ell
Roa
d
Cowie Street
Maunsell Road
Tito
ki
St
reet
Maunsell Road Extension
Cenotaph RoadCenotaph
Ce
no
tap
hR
oa
d
MuseumMuseum Circuit
LittleG
eorgeSt
Foot
ball
Road
Footb
all R
oa
d
WintergardenRoad
WintergardenRoad
RoadRoad
Kioskosk
CresCres
TheThe
centcent
DomainDomain
Drive
Drive
George
Street
George Street
Mo
rga
nStr
eet
Cla
yto
nSt
reet
Alma Street
Sarawia Street
Railway Street
YoungsLane
Laxt
onTe
rrac
e
Broadw
ay
Cresc
ent
Da
vis
Bro
ad
wa
y
Sym
on
ds
Str
eet
Exm
ou
th
St
reet
Basque Road
Du
nd
on
ald
Stre
et
Bu
rlei
gh
Stre
et
Nu
gen
tSt
reet
New North
Road
Boston Road
Water StreetLauder R
oad
Au
bu
rn
S
tree
t
Road
Boston
Sev
ern
Rd
Ara
wa
Str
eet
Ed
wa
rdW
ayt
eP
lace
KhyberPass
Road
Cla
rem
on
t
Str
eet
Pa
rkfi
eld
Terr
ace
Hu
ntl
yA
ven
ue
Su
iter
Stre
et
Kin
gd
on
Str
eet
Short Street
Leek
Str
eet
ea
ux W
ay
Joh
nS
toke
sTe
rra
ce
Ma
ui
Grove
JosephBanks
Terrace
Bro
ad
wa
y
TeedStreet
KentStreet
Osb
orn
eSt
reet
York
Stre
et
EdenStreet
Bo
urk
eSt
reet
MorrowStreet
MortimerPass
Co
ven
try
Lan
e
Balm Street
Nu
ffie
ldLa
ne
Nu
ffie
ldStr
eet
Cro
wh
urs
tSt
reet
SeccombesRoad
Roxburgh Street Mel
rose
Str
eet
McC
oll
Str
eet
Ma
un
ga
wh
au
Ro
ad
Lion Place
Alm
ora
h
Pla
ce
Alm
ora
h
Gill
ies
Ave
nu
e
Remuera
Road
Belmont
Te
rrace
Mid
dleto
n Mam
ieStreetLauriston
Avenue
Ad
aStreet
ElyAvenue
Nikau
Street
Shaddock S
treet
Korari Street
Flower
Street
Ruru Street
Akiraho Street
Sylvan Avenue East
We
Street
Edenvale Cres
Edenvale Cres
Graysons Lane
Puka St
Mount Eden
Ro
ad
Ma
rySt
reet
Ha
rold
Stre
et
Kohekohe
Street
Edwin
Street
EnfieldStreet
Normanby
Roa
d
Norm
anbyR
oa
d
Roa
d
Clive
Clive
Road
Barnett Crescent
Mo
un
tain
Ro
ad
Gilgit Road
Rockwood
Pla
ce
Esp
lan
ad
eR
oa
d
Basset
tRoad
Mid
dle
ton
Ro
ad
So
uth
ern
Motorw
ay
Road
Ba
ssett
Furn
Road
Jam
es
Cook
Crescent
Ro
ad
SeccombesRoad
Mo
un
tain
Ro
ad
KhyberPass
Road
KhyberPass
Road
Mount Eden
Road
Kelly
Aro
tau
Pla
ce
Ma
dei
raP
lace
Burton St
Madeira Lane
NorStreet
Newton Road
Sym
ond
s
Stre
et
LowLow
Dom
ain
Drive
Dom
ain
Drive
erer
Lovers
Lovers
Lane
Lane
Dri
ve
Road
Gra
ndst
and
Roa
dN
orth
Grandstand
South
Hob
son
Str
eet
Alb
ert
Stre
et
Fed
era
l
St
reet Ellio
ttEl
liott
Darby Street
Darby Street
Stre
etSt
reet
Qu
een
Stre
ett
Rutland Street
Rutland Street
Aird
ale
Street
da
leStreet
Govern
or
FitzroyPl
Govern
or
FitzroyPl
Mayoral
Dri
ve
Lorn
eStreet
Lorn
eStreet
Ma
yora
lD
rive
Drive
Drive
Mayoral
Mayoral
Que
enSt
reet
Upp
er
Randolph Street
Sa
int
Ben
edic
tsSt
reet
Street
Karaka
DacreStreet
West Street
St S
th
Fra
nce
Sta
ble
Lan
e
Hohipere Street
Glenside Cres
AlexEvans
Street
Ian Mckinnon Drive
Canada Street
Cross Street
South St
East Street
cury
Lan
e
rM
e
Karan
Qu
een
Stre
et
Up
per
ScotiaPlace
Live
rpool
Stre
et
Turner Street
Qu
een
Stre
et
WaverleyStreet
Marmion Streett
White
White
Stre
et
Stre
et
Pitt S
tree
t
UnionSt
Hopet
ounSt
Vin
cent
Stre
et
Greys
Avenue
Greys
Avenue
Nicholas
Street
Poynton TerraceBeresford Sq
LaneLane
Gra
fton
Gra
fton
Mew
s
Mew
s
st
wich
MahuhuMahuhu
Parnell Rise
Parnell Rise
Security
Information Desk
Hospital
Security
Information Desk
Hospital
Access Parking
Bus Stop (city service)
City-Tämaki Bus Stop
Access Parking
Bus Stop (city service)
City-Tämaki Bus Stop
Traffic Lights
Underpass
Visitor Parking
Traffic Lights
Underpass
Visitor Parking
Pedestrian CrossingPedestrian Crossing
Staff Parking, Area NoStaff Parking, Area No26
P
CITY CAMPUS
GRAFTON CAMPUS
LECTURE THEATRES
LIBRARIES
GU1142
NEWMARKET CAMPUS
Academic es 620Programm I8Academic Services 105 F9Accommodation Solutions 315 G9Accounting & Finance, Dept of 260 G,H10Acoustics Research & Testing Service 422 H9Advancement Office 135 E9Alfred Nathan House 103 F9Alumni Relations 135 E9APEC Study Centre 260 G,H10Applications & Admissions 105 F9Architecture & Planning, School of 421 H9Arts 1 Building 206 G10Arts 2 Building 207 G10Asian Studies 207 G10Auckland Bioengineering Institute 439 I8Auckland Law School 801-803, 810 E10Auckland UniServices Ltd 439 I8Auckland University Press 810 E10Auckland University Students’ Association, AUSA 322 G9Bayreuth House 220 G10Belgrave House 212 G10Biological Sciences, School of 106, 110, 118 F10, G10Biology Building 106 G10Business School 260 G,H10Business Law, Research Centre for 803 E10Business School Computer Laboratories 260 G,H10Call Centre 620 I8Campus Life 315 G9Campus Recreation 314 G9Campus Store 311 G9Careers Services 105 F9Cashiers 105 F9Catholic Tertiary Centre 805 E10Chemical & Materials Engineering, Dept of 401 H9Chemical Sciences, School of 301 H9Civil & Environmental Engineering, Dept of 401 H9Commercial Law, Dept of 260 G,H10Communications 804 E10Comparative Literature, Centre for 206 G10Computer Science, Dept of 303 G,H9Conference Centre 423 H9Confucius Institute 260 G,H10Continuing Education 315 G9Counselling 315 G9Creative Arts & Industries, National Institute of 113, 250,
421-423, 431-433, 804, 820 D9, E,F,G10, H,I9, I8Cultures, Languages & Linguistics, School of 206-207 G10Custodial Services 409 H9Dance Studies 421, 820 D9, H9DELNA 206 G10Disability Services 105 F9Drama Studio 206 G10Economics, Dept of 260 G,H10Education (Liberal Arts Programme) 201 G10Elam School of Fine Arts 113, 431-433 F10, I8, I9Electrical & Computer Engineering, Dept of 301, 303 G9, H9ELSAC 315 G9Engineering Science, Dept of 439 I8English Language Academy 619 I,J7Environment, School of 114, 201 F10, G10Equity Office 119 F9European Languages & Literatures 206 G10Examinations 105& Timetable Services Office (ETSO) ,
620 F9, I8External Relations 135 E9Facilities Management 201 G10Fale Pasifika Complex 273-275 G10Financial Services 620 I8Fine Arts, Elam School of 113, 431-433 F10, I8, I9Fisher Building 804 E10Food Science Programme 301 H9Forensic Science Programme 301 H9George Fraser Gallery 132 F9General Library 109 G9Graduate Centre 119 F9Graduate School of Management 260 G,H10Graduation 620 I8Grounds Maintenance 120-122 F10Gus Fisher Gallery 820 D9Health, Safety & Wellness Manager 620 I8Health Services 315 G9Human Resources 620 I8Human Sciences Building 201 G10Humanities, School of 206-207 G10ID Card Centre 105 F9Information Commons 315 G9Information Systems & Operations Management, Dept of
260 G,H10International Office 104 G10International Relations Office 104 G10iSPACE (for international students) 315 G9IT Services, ITS 260, 409, 435 G,H10, H9, I8James Henare M ori Research Centre 225ā G11John Hood Plaza 260 H10Kate Edger Information Commons 315 G9Kenneth Myers Centre 820 D9Key, Access & Parking Control 409 H9Language Laboratories 201 G10Learning & Research in Higher Education, Centre for (CLeaR)
804 E10Learning Environment Support Unit (LESU) 260 G,H10Legal Research Foundation 801 E10Library, General 109 G9Lippincott Cottage 118 F10Lodge, Old Government House 123 E10Maclaurin Chapel & Chaplains 107 E10Maidment Theatre, Musgrove Studio 313 G9Maintenance Workshops 201 G10Management & International Business, Dept of 260 G,H10M ori Material Culture Workshop 226ā G11M ori & Pacific Studies - Te W nanga o Waipapa 253ā ā 226, G11Marae 251-252 G11Marketing 804 E10Marketing, Dept of 260 G,H10
Mathematics, Dept of 303 G,H9Maurice Wilkins Centre for Molecular Biodiscovery 110 F10Mechanical Engineering, Dept of 401 H9Media Productions 804 E10Mona House 216 G10Musgrove Studio Theatre 313 G9Music, School of 250, 804, 820 D9, E,10, G10Muslim Prayer Room 301 H9Newman Hall 805 E10New Start 206 G10New Zealand Art Research & Discovery, Centre for 421 H9New Zealand Asia Institute 260 G,H10New Zealand Leadership Institute 260 G,H10Ng Pae o te M ramatanga (National Institute of Researchā ā
Excellence for M ori Development & Advancement) 253ā G11Okareta House 215 G10Old Choral Hall 104 G10Old Government House 102 F10O’Rorke Hall 614 I8Owen G Glenn Building 260 G,H10Pacific Studies, Centre for 273 G10People & Organisational Development (POD) Training Team
620 I8Photographers 804 E10Physics, Dept of 303 G,H9Planning & Quality Office 105 F9Polymers & Coatings Science Programme 301 H9Property, Dept of 260 G,H10Property Services 201, 409, 620 G10, H9, I8Psychology, of 201School G10Records, Enrolment & Fees 620 I8Recreation Centre 314 G9Research Office 620 I8Retail Facilities 311, 315 G9Scholarships & Financial Support 620 I8School of Graduate Studies 119 F9Schools Partnership Office 119 F9Science Centre 301 303- G9, H9Shared Services 620 I8Short Courses 260 G,H10Social Sciences, School of 201, 206-207, 408 G10, H10Staff Common Room 102 F10Statistics, Dept of 303 G,H9Strata, Postgraduate Commons 315 G9Student Commons 315 G9Student Contact & Support 620 I8Student Financial Support 620 I8Student Information & Marketing Services 105, 620, 804
E10, F9, I8Student Information Centre 105 F9Student Learning 315 G9Student Recruitment & Course Advice 119 F9Student Union 311-312 G9Students’ Association, AUSA 322 G9Te K hanga Reo 255ö G11Te Wänanga o Waipapa - Mäori Studies & Pacific Studies,
School of 253, 273-275 G10, G11Tertiary Foundation Certificate Programme 206 G10Thomas Building 110 F10Translation Studies, Centre for 206 G10Transport 409 H9Tuition Fees 105 F9UniLodge 813-814 F11Unisafe / Security 409 H9UniServices 439 I8University Careers Services 105 F9
University Hall 440 I8University Hall Apartments 436 I8University Health Services 315 G9University House 135 E9University of Auckland Foundation 135 E9University of Auckland Society 135 E9Vice-Chancellor’s Office 105 F9Whitaker Hall 601-603 J7, K7Women’s Studies 201 G10
Algie ( )Law 801 E10Architecture ( )ALR 421 H9Arts 1 (Arts) 206 G10Arts 2 (C303) 207 G10Biology ( )BLT 106 G10Cell Biology ( io )CMB LT 110 F10Chemistry (Chem) 301 H9ClockTower (ClockT) 105 F9Commerce A ( )CA 114 F10Conference Centre (Conf) 423 H9Engineering (Eng) 401, 403 H9Fine Arts ( )FA 432 I9Fisher & Paykel Appliances Auditorium ( ) 260FPAA G10Human Sciences ( )HSB 201 G10Law (Algie, Northey, Stone)Small, , 803801 E10Library (Lib) 109 G9Mathematics ( ) 303MLT G9Music (M) 250 G10Newman Hall ( ) 805NH E10Northey (Law) 801 E10Old Choral Hall ( )OCH 104 G10Old Government House ( )OGH 102 F10Owen G Glenn Building ( , 260.098)FPAA, 260OGGB G,H10Physics ( ) 303PLT G9Small (Law) 803 E10Statistics ( )SLT 303 G9Stone (Law) 801 E10
MAIN DIRECTORYMAIN DIRECTORY
LECTURE THEATRESLECTURE THEATRES
Architecture 423 H9Engineering 402 G10Fine Arts 432 I9General Library 109 G9Law (Davis) 802 E10Music 250 G10
LIBRARIESLIBRARIES
Arts FO: 215 , SC: 201G10 G10Business & Economics FO: 260 , SC: 260G10 H10Creative Arts & Industries FO: 423 , SC: 421H9 H9Engineering FO: 402 , SC: 402G10 H9Law FO: 801 , SC: 810E10 E10Science FO: 302 , SC: 303I8 G9
The Map Room, Libraries and Learning Services© The University of Auckland, February 2016
The Map Room, Libraries and Learning Services© The University of Auckland, February 2016
FACULTY OFFICES (FO)STUDENT CENTRES (SC)FACULTY OFFICES (FO)STUDENT CENTRES (SC)
WELCOME
We would like to warmly welcome everyone to the ALTAANZ conference 2016 in beautiful Auckland,
NewZealand.Weareparticularlyexcitedasthisisthefirsttimethatthiskeyeventinthecalendarofthe
associationwillbeheldinNewZealand.Wearelookingforwardtosharingideas,furtheringourmissionof
research, training and policy formation and continuing to build a community of language assessment
specialists in our region, and in particular, to learn more about the assessment work of colleagues in
NewZealand.Weareexcitedtoseethatthisyear’sconferencehasattracteddelegatesfrommanyregions
oftheworld,includingcountriesinAsia,Europe,AfricaandNorthAmerica.
Theconferencetheme‘Intheclassroomandbeyond:assessinglanguageabilityindifferentcontexts’reflects
ALTAANZ’saimofconnectingclassroomteachersandresearchers.Thisisreflectedinaspecialdayscheduled
for teachers. We are hoping that these two-way conversations between teachers and researchers will
continue to inform the practices of both groups. We are also happy to see, for the first time at this
conference,astudent-organisedeventandwearehopingthatmanystudentswillusethisopportunityto
networkandmakelastingfriendships.
The program promises high quality presentations around a large range of language assessment-related
topics, including both high-stakes standardised assessments, and formative assessment practices in the
classroom.Ourfourkeynotespeakers,AssociateProfessorMattPoehner(PennStateUniversity),Professor
BarryO’Sullivan(BritishCouncil),DrPeterKeegan(UniversityofAuckland)andDrUteKnoch(Universityof
Melbourne),bringawealthofcombinedexperienceinmanycontextsoflanguageassessment.
Weowespecialthankstothesponsors,IDPIELTS,TOEFL,BritishCouncil,Pearson,theUniversityofAuckland,
Cactuslab,andDELNAfortheirgeneroussponsorshipoftheconference.Thesecontributionsareextremely
importantandmakeitpossibleforayoungorganisationsuchasALTAANZtocontinuetogrowandtokeep
registrationfeesforstudentsandteachersreasonable.
Wewould liketothanktheconferenceorganisingcommittee inAuckland, inparticular thetwoco-chairs
John Read and Janet von Randow. Organising such an event is challenging and takes many months of
planning.
Wehopethatyoufindtheexperiencefruitfulprofessionallyandpersonallyandthatyouhaveapleasantstay
inAuckland.WearealsohopingyouwillbeinspiredtoreturntoAucklandin2017whereALTAANZwillbe
co-organisingaconferencestrandinthejointALAA/ALANZ/ALTAANZconferenceandagainin2018forthe
prestigious Language Testing Research Colloquium, the annual conference of the International Language
TestingAssociationwithwhichALTAANZisaffiliated.
AngelaScarino&UteKnoch
ALTAANZCo-Presidents
ABOUTALTAANZ
The purpose of the Association for Language Testing and Assessment of Australia and New Zealand
(ALTAANZ) istopromotebestpractice in languageassessment ineducationalandprofessionalsettings in
thesetwocountriesandtofostercollaborationbetweenacademia,schoolsandotheragenciesresponsible
forlanguagetestingorassessment.Itsgoalsarelistedunderthreebroadheadingsbelow:
Training:
Stimulateprofessionalgrowthandbestpracticeinlanguagetestingandassessmentthroughworkshopsand
conferences.
Research:
Promoteresearch in languagetestingandassessmentthroughseminars,conferencesand/orpublications
(ALTAANZpublishesaweb-basedjournalandanewsletter).
Policyformation/advice:
Provide advice on assessment to public and other relevant agencies on assessment-related issues, and
advocateonbehalfoftest-takers,studentsandotherstakeholderswhoselifechancesmaybeaffectedby
assessment-relateddecisions.
Forfurtherinformationabouttheorganisation,pleasevisitthewebsiteat:http://www.altaanz.org/.
TobecomeamemberofALTAANZ,pleasedownloadamembershipformfromthewebsiteandemailitto
ALTAANZCommittee
Co-Presidents
DrUteKnoch(UniversityofMelbourne)
AssociateProfessorAngelaScarino(UniversityofSouthAustralia)
VicePresident
AssociateProfessorAekPhakiti(UniversityofSydney)
Secretary
DrKatherineQuigley(VictoriaUniversityofWellington)
Treasurer
DeniseAngelo(TheAustralianNationalUniversity)
PLTAEditors
DrSallyO’Hagan(UniversityofMelbourne)andDrLynMay(QueenslandUniversityofTechnology)
Communicationsofficer(websiteandnewsletter)
DrJohannaMotteram(UniversityofAdelaide)
Studentrepresentatives
XiaohuaLiu(UniversityofAuckland)andMeganYucel(UniversityofQueensland)
ALTAANZCONFERENCECOMMITTEES&GROUPS
ConferenceOrganisingCommitteeCo-chairs:
JohnReadandJanetvonRandow,UniversityofAuckland
KarenAshton MasseyUniversity
JenniBedford UniversityofAuckland
MorenaBotelhodeMagalhães UniversityofAuckland
DarrenConway LanguagesInternational,Auckland
RosemaryErlam UniversityofAuckland
PeterGu VictoriaUniversityofWellington
ShelleeHall
MareeJeurissen UniversityofAuckland
PeterKeegan UniversityofAuckland
MargaretKitchen UniversityofAuckland
UteKnoch UniversityofMelbourne
XiaohuaLiu UniversityofAuckland
KatherineQuigley VictoriaUniversityofWellington
Assistedby:
AmyEdwards EventServices,UniversityofAuckland
AnnemiekHuisman LTRC,UniversityofMelbourne
JohannaMotteram UniversityofAdelaide
MartinvonRandow UniversityofAuckland
BestStudentPresentationAwardCommitteeKatherineQuigley(Chair) VictoriaUniversityofWellington
RosemaryErlam UniversityofAuckland
PeterGu VictoriaUniversityofWellington
MargaretKitchen UniversityofAuckland
AbstractReviewersKarenAshton MasseyUniversity
AnaMariaDucasse RMITUniversity
MartinEast UniversityofAuckland
CathieElder UniversityofMelbourne
RosemaryErlam UniversityofAuckland
KellieFrost UniversityofMelbourne
PeterGu VictoriaUniversityofWellington
MichaelHarrington UniversityofQueensland
KathrynHill UniversityofMelbourne
NorikoIwashita UniversityofMelbourne
PeterKeegan UniversityofAuckland
UteKnoch UniversityofMelbourne
SusyMacqueen AustralianNationalUniversity
TimMcNamara UniversityofMelbourne
SallyO’Hagan UniversityofMelbourne
AekPhakiti UniversityofSydney
JohnPill AmericanUniversityofBeirut
KatherineQuigley VictoriaUniversityofWellington
JohnRead UniversityofAuckland
CarstenRoever UniversityofMelbourne
BestPLTAPaperSelectionCommitteeAekPhakiti(Chair) UniversityofSydney
SusyMacqueen AustralianNationalUniversity
AngelaScarino UniversityofSouthAustralia
RosemaryWette UniversityofAuckland
Winner,BestPLTAPaper2013–2015Knoch,U.,&Elder,C.(2013).Aframeworkforvalidatingpost-entrylanguageassessments(PELAs).
PapersinLanguageTestingandAssessment,2(2),48–66.StudentTravelAwardCommitteeAekPhakiti(Chair) UniversityofSydney
LocNguyen(formerawardwinner) VictoriaUniversityofWellington
SallyO’Hagan UniversityofMelbourne
Winners,StudentTravelAward2016SimonDavidson UniversityofMelbourne
DePhung UniversityofNewSouthWales
StudentVolunteersDoctoralcandidatesfromAppliedLanguageStudiesandLinguistics,UniversityofAuckland.
VincentGreenier
PriscillaShak
VivianQiongWang
MiyoungSong
XiaomingXun
School of Cultures, Languages and Linguistics
IMPORTANTINFORMATION
RegistrationThursday 17thNovember8am–5pminthefoyer,Level0,OGGB
Friday 18thNovember8amonwardsinthefoyer,Level0,OGGB
Saturday 19thNovember8amonwardsinthefoyer,Level0,OGGB
WorkshopsThursday 17thNovember9am–12pmand1–4pm,Level0,OGGB,Rooms040BandComputerLab5
ConferenceFriday18th–Saturday19thNovember8:50am–5:30pm,Level0OGGB
• OpeningReception–Thursday17thNovember5pminthefoyer,Level0,OGGB
• AGM–Friday18thNovember1:25–2:25pm,Level0,OGGB5
• ConferenceDinner–Friday18thNovember6:45pm,Level3,OGGB,DecimaGlennRoom
• Students’lunch–Saturday19thNovember12–1pm,Level0,OGGB,Room008
Guidelinesforpresenters1) Paper presentations – These cover a range of topics and focus on both research and assessment.
Presenters are researchers and practitioners from diverse sectors. Presentationswill be 20minutes,
followed by 10minutes’ discussion. Please save your slides on a USB stick and copy the file to the
computerinyourroomwellbeforeyourtalk.
2) WorkinProgress(WIP)Saturday19thNovember11am–12pm
This sessiongives researchersand teachers theopportunity to shareanddiscussaspectsof research
projects inwhichtheyarecurrently involved,orclassroomassessmenttaskstheyhavedeveloped. In
doingso,theywillreceivefeedbackfromconferenceattendees.Theone-hoursessionwillbedivided
intothreelotsof20minutes,sothateachpresenterhastheopportunitytodiscusstheirproject/task
with3groupsof interested conferenceattendees. Each roomwill have severalpresenters, andeach
presenterwillbestationedatatableinadifferentareaoftheroom.
FacilitiesoncampusATMs:Level1,OGGB,byASB,orStudentCommons,Buildings315and322onthemap,page3of
conferencehandbook:ANZ.ASB.BNZ,Westpac.
PostOffice&Pharmacy–8am–6pmweekdays,KateEdgerCommons,Building315
RecreationCentre(Opentopublic)Building314,SymondsSt.Openinghours:Monday–Thursday6am–
9.30pm,Friday6amto8.30pm,Weekend7am–6.30pmPleasetakeyourconferencenametagtoshowto
getthespecialcasualrateforconferencedelegatesof$10asession.
Carpark(Opentopublic)UnderOGGB,entrancefromGraftonRd.Weekdayearlybird(entrybefore
10:30am,exitby6:30pm)is$12;weekendflatrateis$6.
Internetaccess:informationwillbegiventoyouattheregistrationdeskasyoucollectyourconferencebag.
Pleasenotethatlunchandmorning/afternoonteawillbeservedattheconferencevenueduringtheconferenceonFridayandSaturday.
Cafesoncampus–ExcelCafé,Level1,OGGBShakyIsles(7:30am–8pmweekdaysand9am–4:30pmSaturday)intheKateEdgerCommons,Building315
Tank(8am–7pm)weekdays,KateEdgerCommons,Building315
OurmainCityCampusfoodcourtcatersforarangeoftastes.HelloChinese;UniSushi;UniKebab;JewelofIndia.
Location:AUSAQuad,Building322,�CityCampus.Openinghours:Monday–Friday,8:30am–7pm.
RestaurantsnearthecampusThereareanumberofrestaurantswithavarietyofcuisinesnearthecampus(approx.15minswalkfrom
theconferencevenue)aswellasoncampus.Avarietycanbefoundin:
• TheViaduct,ontheharbourapproximately15minutes’walkfromtheconferencevenue
• Britomart,neartheharbour,a10minutewalkfromtheconferencevenue
• TheStables,ElliotSt,behindSmithandCaugheysonQueenSt
FoodStoreonCampus–Level2KateEdgerCommons,Building315
Supermarketsinthecity–Countdown,19–25VictoriaStWest,(7am–10pm)
NewWorld,125QueenSt(8am–10pm)
Countdown,76QuaySt(24hours)
DevonportwalkSunday20thNovember,2pmferryfromFerryBuilding,QuaySt
Awalk todiscoverDevonport,Auckland’s historic seaside town, bustlingwith cafés, shops and galleries.
Rosemary,aDevonportresidentandALTAANZattendee,willmeetyouattheferryat2:15(yes,itisashort
12minuteboatride).PlantospendaroundanhourdiscoveringhistoricDevonport.Thewalkwill include
scalingthesummit(byroad)ofMtVictoriaforthosewhoarestouthearted(walkingshoesrecommended).
Otherscanexplorethevillage.FerriesreturntoAucklandataquarterpastandquartertothehour.
PRE-CONFERENCEWORKSHOPS–THURSDAY17THNOVEMBER
Registration 8:30am–5pm Level0,OwenGGlennBuilding(OGGB)
Workshops 9am–12pm
Workshop1(040C)
DynamicAssessment:Leveragingclassroomactivitiestounderstandand
supportlearnerlanguagedevelopment MatthewPoehner
Workshop2(ComputerLab5)
IntroductiontoRaschmeasurementusingWinsteps UteKnoch
Lunchbreak 12–1pmFoodanddrinkscanbepurchasedfromthecaféonLevel1,OGGB,oracross
SymondsStreetintheStudentCommons
Workshops 1–4pm
Workshop3(040C)
Usingfreeonlineresourcestodevelopreadingtextsforclassroom
assessment BarryO’Sullivan
Workshop4(ComputerLab5)
Introductiontomany-facetRaschmeasurement UteKnoch
Welcome
reception5–7pm Level0,OGGB
ABSTRACTSWorkshop1: DynamicAssessment:Leveragingclassroomactivitiestounderstandandsupportlearner languagedevelopment MattPoehner–TheUniversityofPennsylvaniaSecond/foreignlanguage(L2)teachersareroutinelychargedwithpromotinglearners’developmentinthe
targetlanguagewhileatthesametimeconductingformalandinformalassessmentsofprogress.Thesesets
ofresponsibilitiesareoftencharacterizedbydifferentpracticesandsetsofassumptions,e.g.offeringsupport
whenlearnersencounterdifficultiesmaybeunderstoodquitedifferentlyduringinstructionversusassessment.
InDynamicAssessment(DA), teachingandassessingareunderstoodas interrelatedfeaturesofthesame
activity,onethatseekstopromotethedevelopmentoflearnerL2abilities.Byjointlyengagingwithlearners
inactivitiesthatarebeyondtheircurrentindependentfunctioning,itispossibletoreachadiagnosisoftheir
emerging abilities; that is, one can glimpse abilities that have not yet fully developed but that are still
ripening.Atthesametime,theinstructionalqualityofthisinteractionmayservetocontinueguidingtheir
development.
Thisworkshopisconcernedwithhowbothateachingandanassessmentfunctionmaybepursuedduringa
givenactivity.ParticipantswillexamineinstancesofDAinteractionstoidentifysomeofthewaysinwhich
learnerabilitiesmaymanifest.Principlesofprobingandpromptingthroughinteraction,systematicity,and
shiftingfocusfromindividualtogroup,willbehighlighted.Inaddition,participantswillbeinvitedtobeginto
planhowactivitiesfromtheirteachingcontextsmightbereorganizedaccordingtoDAprinciples.
Workshop2: IntroductiontoRaschmeasurementusingWinstepsUteKnoch–TheUniversityofMelbourne
ThisworkshopaimstoprovideparticipantswithanintroductiontothebasicRaschmodel.Thesessionwill
includeamixtureoftheoryandhands-onpractice.Differencesbetweenclassicalandmoderntesttheorywill
beexplored.TheuseofthestatisticalsoftwareWinstepswillbedemonstrated,andparticipantswillhave
plentyofopportunitytogainhands-onexperiencewiththesoftwareusingdataprovidedintheworkshop.
TheinterpretationoftheoutputofaRaschanalysiswillbeoneofthekeyfociofthesession.Participants
maybring theirowndata sets to theworkshop.Nopriorknowledge is requiredandparticipantsarenot
requiredtohaveanunderstandingofstatisticsormathematics.
Workshop3: UsingfreeonlineresourcestodevelopreadingtextsforclassroomassessmentBarryO’Sullivan–TheBritishCouncil
Selectingappropriatetextsforuseintestsandforclassroomactivitiesisacriticalpartofateacher’swork.
It’sprettyeasytothinkaboutsomeoftheissuesthatneedtobeconsidered:length,topic,difficulty.Thefirst
twooftheseareeasilydealtwith.Countthewords.Knowyourstudents.Thethird,however,causesteachers
themostgrief.Howcanweknowinadvancehowdifficultthestudentsarelikelytofindthetext?Unlesswe
canpredict,evenroughly,thedifficultyitbecomesextremelyproblematictouseatextmeaningfullyinatest.
Somelanguagetestingcompanieshavebeendealingwiththisprobleminasystematicwayforyears,others
stilltakea“we’retheexperts,wejustknow”approach.Inmyownwork, Idon’t liketoleaveanythingto
chanceandliketohaveasmuchinformationaboutatextasIpossiblycanbeforeIevenconsiderusingitin
atest. Iverymuchbelievethatweshouldworkhardtoestablishsomemeasuresoftextstohelpbuilda
pictureofitslikelydifficulty.Luckily,thereareanumberofresourcesavailabletotheteacherandtesterthat
arefreeandgenerallyquiteeasytouse.
Inthisworkshop,wewillworkwithanumberoftheseresourcestobuildausefulpictureofwhatanideal
textshouldlooklike.Usingmaterialssuppliedbyparticipants,wewillcreateaspecificationtemplatethatis
targetedatspecificclassesorgroupsoflearners.Participantsareaskedtobringalonganumberoftextsthat
havebeensuccessfullyusedforaspecificclassorlevelinthepast(4to6wouldbegood)asthesewillbe
usedtobuildthespecification.
Workshop4: Introductiontomany-facetRaschmeasurementUteKnoch–TheUniversityofMelbourne
Thisworkshopaimstoprovideparticipantswithanintroductiontomany-facetRaschmeasurementusing
Facets.Thesessionwillincludeamixtureoftheoryandhands-onpractice.Differencesbetweenclassicaland
moderntesttheorywillbeexplored,withaparticularfocusonunderstandingtheeffectsraterscanhaveon
theoutcomesof anassessment. Theuseof the statistical softwareFacetswill bedemonstratedand the
interpretationoftheoutputwillbeexploredusingdataprovidedintheworkshop.Participantsarealsoable
to bring their own data sets to the workshop. No prior knowledge is required and participants are not
requiredtohaveanunderstandingofstatisticsormathematics.
13
CONFERENCEDAY1–FRIDAY18THNOVEMBERRegistrationopen 8–8:50am Level0,OwenGGlennBuilding(OGGB)Welcome 8:50–9:15am OGGB051Plenaryaddress1,9:15–10:15am,MattPoehner:DynamicAssessmentandVygotsky’sunrealizedvisionofdevelopmentaleducation(OGGB5) StreamA:CaseRm2 StreamB:CaseRm3 StreamC:CaseRm4 StreamD:OGGB5 Secondlanguageproficiency
modelsandimplicationsfordevelopingassessmenttasks
Standardisedtests:development,implementationand/orusebyinstitutionalstakeholders
Assessmentforthelearningofindigenouslanguages
Assessinglanguageforacademicpurposes
ParallelsessionA 10:20–10:50am KarenHuangDevelopingaChineseplacementtestforheritagestudentsintertiaryeducation:Issuesandconcerns
JessicaWuEvaluatingscorereportingpracticefortwolarge-scaleEFLtests:Intendedgoalandactualuse
PeterKeeganTriallingaMāorilanguagepronunciationtoolbasedonaMāorispeakerdatabase
PamelaHumphreysTheoreticalandconceptualmodelsofacademicEnglishlanguageproficiencyinhighereducation:ConsiderationsforprincipledassessmentinEAP
Morningtea,10:50–11:15amParallelsessionB 11:20–11:50am RuslanSuvorov
Test-takingstrategiesduringthecompletionofmultiple-choiceitemsfromtheMichiganEnglishTest:Evidencefromeyetrackingandverbalreports
PaulMooreCohesioninorallanguagetestperformance
JeanetteKingTuhingaMāhorahora:acorpusofchildren’swritinginMāori
KellieFrost,UteKnoch,AnnemiekHuismanSettingstandardsonapost-entrylanguageassessment:Exploringdifferencesinvaluesofcontentlecturersandacademicskillsstaff
Secondlanguageproficiencymodelsandimplicationsfordevelopingassessmenttasks
Standardisedtests:development,implementationand/orusebyinstitutionalstakeholders
Classroom-basedassessment:issuesandpractice
Assessinglanguageforacademicpurposes
ParallelsessionC 11:55am–12:25pm LinLinInvestigatingrelationshipsbetweensecondlanguagetesttakers’strategyuseandChinesereadingcomprehensiontestperformance
FawziAlGhazaliInvestigatingtheWashbackEffectonLanguageProficiency:ACaseStudyfromanArabContext
AnneMoirScottLocatingtheLearning:MeasuringtheimpactofL1reflectiononL2development
LeilaIranmaneshEvolutionofformativeassessmentinanEnglishacademicwritingclass:Theroleofemotionandpower
14
Secondlanguageproficiencymodelsandimplicationsfordevelopingassessmenttasks
Standardisedtests:development,implementationand/orusebyinstitutionalstakeholders
Classroom-basedassessment:issuesandpractice
Assessinglanguageforacademicpurposes
ParallelsessionD 12:30–1pm MeganYucelNarrativeinquiryinlanguageassessmentresearch
RosemaryErlamUsingevaluationtopromotechangeinlanguageteacherpractice
NaokiIkedaAssessingL2learners’oralpragmaticandinteractionalabilitiesforuniversitysettings:Implicationsforclassroomassessment
Lunch,1–2:25pm,Level0;ALTAANZAGM,1:25–2:25pm,OGGB5ParallelsessionE 2:30–3pm MikiTokunaga
EffectoftimepressureongrammaticalityjudgmenttestswithL1translation
JanEyreStartingPointsListening:AnonlineassessmentforbeginningEnglishlanguagelearners
PeterDavidsonAssessingEAP:Thecaseforauthenticassessment
XiaohuaLiuAnalysingexistingreadingtesttasks:Implicationsfordevelopingtaskstomeasuredifferentreadingabilities
ParallelsessionF 3:05–3:35pm JinsongFanFactorstructureandfactorialinvarianceofauniversity-basedEnglishtest:Alongitudinalstudy
PeterGuCreatingandvalidatingtheClassroomAssessmentConfidenceIndexamongChineseEFLteachers
MichaelMersiadesValidityofEAPreadingtestinferenceitems:Apilotstudy
Standardisedtests:development,implementationand/orusebyinstitutionalstakeholders
Assessinglanguageforprofessionalpurposes
Assessinglanguageforacademicpurposes
ParallelsessionG 3:40–4:10pm MichelleCzajkowskiJudgementsofwritingproficiencybynon-nativeandnativeEnglishspeakingteachers:Comparingholisticandanalyticalscoring
SusyMacqueenProfession-specificlanguagestandards:Perspectivesfromprofessionalbodiesontheuseoflanguagetests
ShahrzadSaif,ZahraMahdaviLanguageneedsofinternationalgraduatestudentsworkingasteachingassistants(ITAs)inCanadianFrancophoneuniversities:ImplicationsforAssessment
Afternoontea,4:10–4:25pmPlenaryaddress2,4:30–5:30pm,PeterKeegan:MāorilanguagetestingandassessmentinAotearoa:past,presentandfutureprospects(OGGB5)Groupphotos,5:30–5:45pmConferencedinner,6:45onwards,Level3,OGGB,DecimaGlennRoom
15
CONFERENCEDAY2–SATURDAY19THNOVEMBERRegistrationopen 8–8:50am Level0,OwenGGlennBuilding(OGGB)Welcome 8:50–9am HousekeepingPlenaryaddress3,9:00–10:00am,UteKnoch:Measuringwritingdevelopment:Implicationsforresearchandpedagogy(OGGB5) StreamA:CaseRm1 StreamB:CaseRm2 StreamC:CaseRm3 StreamD:CaseRm4 Classroom-basedassessment:
IssuesandpracticeStandardisedtests:development,implementationand/orusebyinstitutionalstakeholders
Expandingexisting,andcreatingnew,validityframeworksforlanguageassessment
Classroom-basedassessment:Issuesandpractice
ParallelsessionH 10:05–10:35am LaurieLuComingtogripswithtechnicalissuesindevelopingandimplementingEAPUnitstandardsassessments
MatthewsMMakgamatha,KathleenHeughMultilingualAssessment:opportunitiesforteacherdevelopmentandequitablelearning
SherylCookeSelectingthegatekeepers:thefairnessdimensionoflanguageproficiencyrequirementsforlanguageassessors
DePhungWhatdidEAL/Dteachersactuallythinkanddowhenmarkingoralperformances?
Morningtea,10:35–10:55am StreamA:CaseRm1 WorksinProgressI:
Seminar40BWorksinProgressII:Seminar40C
RoomDCaseRm4
ParallelsessionI 11–11:30am BernadetteBarkerAssessingLanguagethroughtasksintheclassroom:AprocessfordeterminingwhetherastudenthasadditionalneedstolanguagelearninginEnglish
ApisakSukyingAnInvestigationofReceptiveandProductiveAffixKnowledgeanditsRelatednesstoVocabularySizeinThaiEFLLearners
WenjingYaoLanguageassessmentversuslanguagetesting-acomparativestudyoflanguageassessmentcoursesinNewZealandandChina
MikiTokunagaComparinggrammarknowledgeandproductionofJapaneseEFLlearners
MorenaDiasBotelhodeMagalhãesInvestigatinguseofascreeningtoolforrecommendationsregardingcompulsoryacademicEnglishlanguagecourses
KarenAshtonIt’sajugglingact:Assessinglearnersinthemulti-levellanguageclassroom
ParallelsessionJ 11:35am–12:05pm SueEdwardsIssuesforNewZealandprimarymainstreamteachersassessingEnglishlanguagelearners
FaisalFaisalAssessingwriting:acertifiedteacher’sperception(apreliminaryresearchfinding)
Lunchandstudents’meeting,12:05–1pm
16
StreamA:CaseRm1 StreamB:CaseRm2 StreamC:CaseRm3 Teachers’Forum:OGGB5 Classroom-basedassessment:
IssuesandpracticeAssessinglanguageforprofessionalpurposes
Expandingexisting,andcreatingnew,validityframeworksforlanguageassessment
SeePage17fordetails
ParallelsessionK 1:05–1:35pm KeikoNakaoReflectiveassessmenttasksforinterculturallanguagelearninginabeginnerforeignlanguagecourse
SimonDavidsonCandoctorssetvalidstandardsonanESPtestforhealthprofessionals?
AlbertWeidemanTherefinementoftheideaofconsequentialvaliditywithinanalternativeframeworkforresponsibletestdesign
ParallelsessionL 1:40–2:10pm CateGribbleEnglishlanguageproficiencyandemployment,migrationandprofessionalregistrationoutcomesinhealthcareandearlychildcareeducationinAustralia
ChristinaJudyFernandezTesttakers’speakingstrategies:“Ido,IthinkandIthinkaboutmythoughtsbecause…”
ParallelsessionM 2:15–2:45pm PaulCrumpFitnesstopractise:RevisingtheOccupationalEnglishTestListeningcomponent
KazuoAmmaPartialscoringofsequencingtaskswithdistancepenalty
Afternoontea,2:45–3:10pmClosingplenary(address4),3:15–4:15pm,BarryO’Sullivan:Makingconsequencehappen(OGGB5)Conferenceclosing,4:15–4:35pm
17
TEACHERS’FORUM–SATURDAY19THNOVEMBERThissessiononSaturday,1:05–2:45pminOGGB5willseepractisingprimaryandsecondaryschoolteachersraisingassessmentissuestheyfaceintheirday-to-dayworkwithEnglishLanguageLearners.JanTagaloa(PrimaryBilingual)Howshouldstandardizedassessmentsbecarriedoutinbilingualsettings?JacquiLindsayandIngeMillard(MainstreamPrimary)HowcanweaccessorconductbilingualorotherassessmentstoidentifyspecificlearningneedsofEnglishLanguageLearners(ELLs)inmainstreamsettings?SimonCrosby(SecondaryESOL)WhatassessmentaccommodationsforELLsinmainstreamsettingsarevalidandpracticable?RosemaryGillies(PrimaryESOL)WhatrecommendationswouldyoumakeforinitialplacementassessmentfornewlyarrivedELLs?JenniBedford(inconsultationwithprimaryandsecondaryteachers)WhatneedstobetakenintoconsiderationwhencreatingreadingassessmentsforELLs?Researchers,languageassessmentspecialistsandMinistryofEducationrepresentativeswillrespondtotheseissuesfromavarietyoftheoreticalandpracticalperspectives,andtherewillbetheopportunityforfollow-updiscussionfromthefloor.TheforumwillbechairedbyMareeJeurissen,PresidentofTESOLANZ,JenniBedfordandMargaretKitchen.Inaddition,therewillbetwostrandsofparallelsessionsforteachersonSaturdayfrom10:05amuntil12:05pm:Classroom-basedassessment:Issuesandpractice(RoomA,CaseRm2;andRoomD,OGGB5).
18
PLENARIESPlenary1:“DynamicAssessmentandVygotsky’sunrealizedvisionofdevelopmentaleducation”MatthewE.Poehner,ThePennsylvaniaStateUniversityDay:Friday18thNovember,2016Time:9:15–10:15amRoom:OGGB5DynamicAssessment(DA)referstotheadministrationofanassessmentinwhichtheconventionalapproachofobserving learnersas they independentlycomplete tasks isabandonedand theassessor,ormediator,interveneswhen learnersexperiencedifficulties toofferprompts, feedback, leadingquestions,andotherformsofsupport.Therationalebehindthisdeparturefromacceptedassessmentpracticeisthatthedegreeofexternalsupportlearnersrequiretoovercomeproblemsrevealstheextenttowhichrelevantabilitieshavebeguntodevelop.Inshort,learnerswhofailindependentlybutaresuccessfulwithminimalinterventionaredevelopmentallymoreadvancedthanthoserequiringmoreintensivesupport.ProponentsofDAarguethatitthusprovidesamorenuancedpictureoflearnerabilitieswhilealsopointingtotheformsofsupportthatwere most beneficial to individuals, thereby offering a starting point for subsequent instruction(e.g.Feuerstein,Falik&Feuerstein2015).
Fornearlyhalfacentury,DAhasbeenpursuedinpsychologyandcognitiveeducationwithawiderangeofpopulations (Lidz&Elliott 2000; Sternberg&Grigorenko2002), and formore thanadecade it hasbeenundertaken in L2 educational contexts (Lantolf & Poehner 2014). Despite its considerable promise andextensive research literature, DA has yet to become a fixture of mainstream education. In this paper,IproposethattwoissuesinparticularhaveimpededrealizationofDA’spotentialandmustbeaddressed.Thefirstderivesfromtraditionaldivisionsbetweenformaltestingandday-to-dayclassroomteachingandlearning.OutsideoftheL2field,DAhasprimarilybeenappliedbyassessmentspecialists,withtheresultthatinsightsgainedfromproceduresfrequentlydonotleadtochangestoteachingpractice(seeHaywood&Lidz2007;Tzuriel2011).
Asecondproblem,whichpertainsequallytogeneraleducationandL2teaching,concernstheuseofDAtotarget development of learner abilities in contexts where the curriculum is not guided by a theory ofdevelopment but instead emphasizesmemorization and skills. Following anoverviewofDA’s theoreticalorigins in L. S. Vygotsky’s writings (1987, 1998), I argue that engagement with the Zone of ProximalDevelopment as a framework for cooperative educational activity offers a way forward. Examples arepresentedofDAconductedinbothL2formaltestingandclassroomlearningsituations,withdiscussionofhowthesemayfunctionintandemtocontinuallymonitorlearnerprogress.Inaddition,recentresearchinthe area of L2 Mediated Development (Poehner & Infante 2015, 2016) is highlighted to capture howcurricular revisionsmight further learner appropriation of knowledge about the language in an effort toenhancetheircapacitytoregulatetheirL2use.BiographyDrMatthew E. Poehner is Associate Professor ofWorld Languages Education and Applied Linguistics atThePennsylvaniaStateUniversity(PennState).HehastaughtFrenchasaforeignlanguageandEnglishasasecondlanguageinU.S.K-12schoolsaswellasatprivateinstitutionsanduniversities.AftercompletinghisPhDinAppliedLinguisticsatPennState,hehascontinuedtoworkatthatuniversity.Inhiscurrentposition(since2008)hedirectstheteachereducationprogramforcandidatespursuingcertificationtoteachaworldlanguage in the K-12 school system and also contributes to the doctoral programs in Curriculum andInstructionandAppliedLinguistics.
DrPoehner’sresearchexaminestheuseofSocioculturalTheory,asconceivedbyRussianpsychologistL.S.Vygotsky,asabasis for second languageeducationalpractices, includingDynamicAssessment,MediatedDevelopment,andSystemic-theoretical Instruction.MuchofDrPoehner’sworkhasfocusedspecificallyon
19
DynamicAssessmentas a framework for organizing interactionswith learners in order to simultaneouslydiagnosetheirabilitiesandpromotetheircontinueddevelopment.Hisresearchhasinvolvedpartnershipswithlanguage teachers, learners, and program directors and has been supported through grant awards,particularlythroughtheCenterforAdvancedLanguageProficiencyEducationandResearch(CALPER)atPennState,whichisfundedthroughtheU.S.DepartmentofEducation.Morerecently,DrPoehnerwasco-principalinvestigator for a project funded by the U.S. Department of Education to develop computerized tests ofcomprehensioninChinese,French,andRussianusingprinciplesofDynamicAssessment.
DrPoehneriswidelypublishedintheareasofsecondandforeignlanguageteaching,languageassessment,theoriesofsecondlanguageacquisition,andappliedlinguistics.Heistheauthor,co-author,oreditoroffourbooks,includingSocioculturalTheoryandthepedagogicalimperativeinL2education:Vygotskianpraxisandtheresearch/practicedivide(2014),whichreceivedthe2015KennethW.MildenbergerPrizefromtheModernLanguageAssociation.Anearlierbook,DynamicAssessment:AVygotskianapproachtounderstandingandpromotingsecondlanguagedevelopment(2008),wasafinalistfortheOutstandingBookAwardthroughtheBritishAssociationforAppliedLinguistics.DrPoehnerhaspublishedmorethanthirtyscholarlybookchaptersand journal articles. His work has appeared in venues including TESOL Quarterly, Language TeachingResearch, The Modern Language Journal, Language Testing, and the International Journal of AppliedLinguistics.In2008,hereceivedthePimsleurAwardforOutstandingResearchContributionfromtheAmericanCouncilontheTeachingofForeignLanguages.
20
Plenary2:“MāorilanguagetestingandassessmentinAotearoa:past,presentandfutureprospects”PeterKeegan,UniversityofAucklandDay:Friday18thNovember,2016Time:4:30–5:30pmRoom:OGGB5Despitea longhistoryofteachingMāoriasasubjectandre-introducingMāoriasamediumofeducationsince the late 1980s, there have been few developments of robust tools for assessingMāori language.Theonlystandardized instrument is thee-asTTleMāorinumeracyand literacyonlineassessmenttool forMāori-mediumstudentsinthecompulsoryschoolsector.ArecentdevelopmentistheMinistryofEducationsponsoredKaiakaReoMāoriorallanguageproficiencytool.However,mostprojects,includingtheUniversityofAuckland’slongitudinalstudy‘GrowingUpinNewZealand’,havehadtoadaptexistingtoolsformeasuringtheproficiencyofyoungerspeakersofMāori.
ThispresentationwillprovideanoverviewofrecentMāori languagetestingandassessmentinAotearoa/NewZealand.Despitegovernmentandcommunityefforts to increasethenumbersofspeakersofMāori,Censusresultsclearlyindicatethatthelanguageisdeclining.FormanyMāori-mediumstudents,theschoolremainstheonlydomainwhereMāoriisusedexclusively;homeandcommunityactivitiesformosttendtobeconductedinEnglish.Thismeansthatitisdifficulttodefinewhatrepresentsfirst(or“native”)languageproficiency inMāori foryounger learners.AlthoughMāoridialectsshowvery littlevariation linguistically,manysecondlanguagelearnershavebeguntoinfusetheirpronunciationandwrittenMāoriwithfeaturesthatarecharacteristicofaparticulartribeorregion.However,mostoftheMāorimaterialsproducedtendtofollowadefactostandardizedMāori.ThepresentationdescribesthetoolsthathavebeendevelopedforassessingMāori,includingworkinprogress.Itconcludeswithadiscussionofongoingissues,suchasalackof developers/practitioners with appropriate technical knowledge, and suggests priorities for futuredevelopment.BiographyDrPeter J.Keegan (Waikato-Maniapoto,NgātiPorou) isa senior lecturer inTePunaWānanga (schoolofMāoriEducation),theFacultyofEducationandSocialWork,theUniversityofAuckland,NewZealand.Hisresearch interests include the structure, changes and current use of Māori language, assessment/measurementandlanguagetestingespeciallyinindigenouslanguagecontexts,Māori/indigenousmediumeducation and indigenous literatures. He worked as a project manager for asTTle (assessment tools forteaching and learning), aNew Zealand computer based online numeracy and literacy assessment tool inEnglishandMāori(http://e-asttle.tki.org.nz/).Heteachescoursesonassessmentforteachingandlearning,and serves on government advisory panels on assessment. Peter is a co-editor of Teachers voyaging inplurilingual seas: Young children learning through more than one language (New Zealand Council forEducationalResearch,2016).CurrentresearchprojectsincludedevelopinganonlineMāorilanguagespeechpronunciationaid(MPai)andtryingtovisualizeMāorilanguagecensusdata(http://peterjkeegan.github.io/).Other interestsandactivities includespending timewithwhānau, reading,computers,kayaks,hikingandtryingtokeepfit.
21
Plenary3:“MeasuringL2writingdevelopment:implicationsforresearchandpedagogy”UteKnoch,UniversityofMelbourneDay:Saturday19thNovember,2016Time:9–10amRoom:OGGB5L2writingdevelopmenthasreceivedbothimplicitandexplicitattentionindifferentareasofsecondlanguageresearchsuchassecondlanguageacquisitionandL2writingpedagogyformanyyears,althoughthedifferentresearchstrandsoftendonotoverlapmuchintermsofthedefinitionsusedandthemethodologicalchoicesmade.Manystudieshavenarrowlyfocussedonlinguisticvariables,suchasthedevelopmentofaccuracy,fluencyandcomplexity.Inarecenteditedvolume,Manchon(2012)callsforabroaderconceptualisationofwritingdevelopment,examiningbroaderaspectsinwritingsuchasdiscoursestructures,contentandgenreknowledge.
In this presentation, Iwill focuson the kindofwork thathasbeenundertaken in the areaof L2writingdevelopmentbothinresearchandinclassroomcontexts.Bydrawingonarangeofstudies,Iwillshowthatthereareseveralpossiblespheres inwhichwritingcandevelop.Therearealsoanumberofpurposesformeasuringwritingdevelopment.Iarguethatunlessthemethodologychosenmatchesthesphereofwritingdevelopmentandthepurposeofmeasuringdevelopment, themeasurementwillhave limitations for thestakeholders. Iproposethatconceptualizingwritingdevelopment inthisway,willhelpclarifyoperationaldefinitions applied and tighten measurement designs employed and ultimately broaden the type ofinvestigationsundertakeninbothresearchandeducationalsettings.BiographyDrUteKnochistheDirectoroftheLanguageTestingResearchCentreattheUniversityofMelbourne.Shehaspublishedwidelywith over 40 peer-reviewed publicationswhich have been published in journals such asLanguageTesting,LanguageAssessmentQuarterly,TESOLQuarterly,AppliedLinguistics,AssessingWriting,JournalofSecondLanguageWritingandEnglishforSpecificPurposes.Herresearchinterestsareintheareaofwritingassessment,ratingprocesses,assessinglanguagesforacademicandprofessionalpurposes,andplacementtesting.SheiscurrentlytheCo-PresidentoftheAssociationforLanguageTestingandAssessmentof Australian and New Zealand (ALTAANZ) and has served on the Executive Board of the InternationalLanguage Testing Association (ILTA) from 2012 to 2015. In 2014, Dr Knoch was awarded the TOEFLOutstanding Young Scholar Award by the Educational Testing Service (Princeton, US), recognizing hercontributiontolanguageassessment.
22
Plenary4:“MakingConsequenceHappen”BarryO’Sullivan,TheBritishCouncilDay:Saturday19thNovember,2016Time:3:15–4:15pmRoom:OGGB5Considerationof thesocial consequencesof testusehasbeenacentral theme invalidation theorysinceMessick(1989)broughttheideaintohismodelofvalidity.Whilethenegativeimpactoftestusehasquiteoftenbeenstressed,littlemeaningfulattentionhasbeenpaidtohowtestdevelopersmightoperationalisetheconceptofconsequenceinthetestdevelopmentprocess.Whereconsequencehasbeenaddressed,ithastendedtobeasanaposteriorievidencesource,primarilyconcernedwithtestimpact.Therealityiswedonotknowwhatconsequencemeanstotestdevelopment.
InthispaperIwillfirstoutlinehowthesocialcognitivevalidationmodelhasbeendevelopedoverthepastdecadeormore,describinghowithasinformedtestconceptualisation,developmentandvalidation.Whiletheearlierversionsofthemodelprovedtobeofpracticalusetotestdevelopers,itfailedtorecognisetheimportanceandplaceofconsequenceintheprocess.ThisisparticularlyclearinthewayinwhichWeir(2005)conceptualisedwhathe,andothers,referredtoasconsequentialvalidityasoneofthefinalelementstobebroughtintoplayindevelopmentandvalidation.
OvertimebothWeirandO’Sullivanhaverevisitedthemodel,andinthelatter’smostrecentinterpretation(2014,2016)finallyattemptedtooperationaliseconsequenceinameaningfulway.Thisversionofthemodelseesconsequenceasbeingspecificallyrelatedtothecontextoftestuse,whichitselfisdefinedbythekeystakeholdergroupswhocomprisethatcontext.Inordertounderstandhowthecontextsimpactonthetest,itisnecessarytotakerelevantstakeholdersintoaccountwhenconceptualisingthetestitself.Thishastheeffectofinformingushowtestconstructistobeoperationalised.Itwillalsoinformallofthedecision-makingthatismadeintheprocessoftestdevelopment.
Finally, it will impact on how validation evidence is presented. This latter is critically important, sincetraditionallyvalidationargumentshavebeenwrittenwithnospecificaudienceinmindorwereaimedatanacademicaudience—or,sinceKane(1992),atalegalone.
Byconceptualisingconsequenceinthewaysuggestedherewemustacceptthatvalidationargumentsshouldbetargetedsquarelyatawholerangeofspecificstakeholdergroups.Thiswillimpactonstructure,contentanddeliverymode.Examplesofhowthisisdealtwithinanoperationalwaywillbepresentedanddiscussed.BiographyProfessorBarryO'SullivanistheHeadofAssessmentResearch&DevelopmentattheBritishCouncil,London.HisrecentworkincludesthedevelopmentandvalidationofanewbusinesstobusinesslanguagetestcalledAptis.He is the foundingpresidentof theUKAssociationof LanguageTestingandAssessmentandholdshonorary and visiting chairs at theUniversities of Reading and Roehampton in theUK, theUniversity ofTechnologyMARA(KualaLumpur)andattheUniversityofLisbon.Barry isparticularly interestedin issuesrelated to performance testing, test validation, test-data management and analysis and scaling andcalibration;hehasconductedresearchintofactorsaffectingspokenperformance,assessingraterbehaviour,assessing speakingandwriting, specificpurposeassessment,benchmarkingEnglish language tests to theCommonEuropeanFrameworkofReferenceforLanguagesandstandardsettinginprofessionalcontexts.
Barry’spublicationshaveappearedinanumberofinternationaljournalsandhehaspresentedhisworkatinternationalconferencesaroundtheworld.Hisbooksinclude:IssuesinBusinessEnglishTesting(Cambridge,2006);ModellingPerformanceinOralLanguageTesting(PeterLang,2008);LanguageTesting:TheoriesandPractices (PalgraveMacmillan,2011)andTheCambridgeGuide toSecondLanguageAssessment (withC.Coombe,P.Davidson,andS.Stoynoff,eds.)(Cambridge,2012).Heiscurrentlyfinalising(withCyrilWeir)amajorprojectdocumentingahistoryoflanguagetestingwithintheBritishCounciltobepublishedbyEquinoxin2017,andisworkingontwovolumes(onvalidityandlocalisation)withMichelineChalhoub-DevillefortheBritishCouncilMonographsSeries(alsoEquinox).
23
INDIVIDUALPAPERABSTRACTSFawziAlGhazali(AbuDhabiUniversity,UnitedArabEmirates)
Investigatingthewashbackeffectonlanguageproficiency:AcasestudyfromanArabcontextDay:Friday Time:11:55am–12:25pm Room:CaseRm3Languagetestsarenecessarytogaugelearners’progressandtosetfutureplansforimprovinglanguageskills.Testssometimeshavegreatinfluenceonteachingandlearningthataretailoredtomatchwiththewaytheyaredraftedthroughaphenomenonknownasthe“Washbackeffect”(Wall&Alderson1993).However,whatare the most optimal ways to minimize the negative outcomes of the washback effect on languagedevelopment?
Thispaper reveals thatdiligentanalysisof learners’performance in languagecoursesshowsverystrikingresultsrelatedtotheinfluenceoftestingonteaching.Thecasestudypresentedinthispaperinvestigatestheperformance of some Arab learners in language courses after having graduated from high schools. Theanalysis shows theygethigh scoreson school-level-relatedexamsonwhich theyhaveextensive trainingthroughoutthewholeyears;nonetheless,theyachieveverypoorlyonproficiencytestslikeIELTSorTOEFL.Thisissimplybecauseachievementtestsmeasurehowmuchastudentlearntinaparticularcourse;whereasdoingwellinproficiencytestsdependsonlearners’generalmasteryoflanguage.
Achievementtestspossiblyhavefacevaliditywhentheymeasurewhattheyaimedtomeasure.However,intermsofreliability,thereisnoguaranteethatlearnerswouldbeabletoachievethesameresultseveniftheytakethesametestsafteraperiodoftime.Thissurfaceapproachtolearningcouldhardlyleadtopermanentknowledgeandlearnersareexpectedtostumblebehindwhenfluentcommunicationinaforeignlanguageisneeded.
Thepresenterwillreviewtheseissuesandthepresentationwillprovideinsightsandpedagogicalimplicationsintohowtousetestsinthemostoptimalways.Theaudiencewillbeinvitedtosharetheirideasandreflectontheirexperienceoflanguageassessment.
KazuoAmma(DokkyoUniversity,Japan)Partialscoringofsequencingtaskswithdistancepenalty
Day:Saturday Time:2:15–2:45pm Room:CaseRm4Inlanguagetesting,reorderingitems(words,sentences,andparagraphs)isoneofthecommontechniquesofconfirminglearners’comprehensionaswellasreproductionoftext/discourse.Yetavalidpartialscoringmethodhasnotbeenestablishedtodate.The‘allornothing’method,whichgivesafullscoreonlywhenallelementsarecorrect,iswidelyusedatschoolandbynationwidetests,butitundulydiminishestheconstructvaliditywhenpartofthesequenceiscorrectlyarranged.Avalidscoringmethodshouldreflectthecomplexityofthecognitivemanipulationinvolvedintheproblem-solvingprocess.
Thepresenter’smethodMaximalRelativeSequence(MRS)calculatesthenumberofrelativetransitionsofelementsintheresponsetogivelongestsequenceoftransitions(Amma,2007;2010a;2010b).Forexample,inaresponse“A_C_B_D”thelongesttransitioniseither“A_C_D”or“A_B_D”(score=2).ThisisequivalenttoatechniqueknownasMinimalEditDistance(MED),orcountingtheelementsofdigressions(BorC)fromthecorrectsequence,subtractedfromtheperfecttransitionsize(3).
ComparedwithMED,thealgorithmofMRSisfarsimpler,thussavingtimetocalculatealargenumberofsamples. However,MRS does not consider the factor of distance between the right position and initialincorrectposition.Asaresult,twosequences,“B_A_C_D”and“B_C_D_A”arebothscoredas2(takingup“B_C_D”astheMRS),eventhoughonehastoretrievethecorrectanswerbymovingAoneslotaheadinthefirstsequenceandthreeslotsinthesecond.
Thepresentreportdemonstratesanewcomputerprogramthatgeneralisesthedistancepenaltybycountingtheshiftsforretrieval,thusachievingamoreaccurateandfairmeasurementthanMRS.Anexhaustivelistofscorescomparingvariousmethodswillalsobepresented.
24
KarenAshton(MasseyUniversity,NewZealand)It’sajugglingact:Assessinglearnersinthemulti-levellanguageclassroom
Day:Saturday Time:11–11:30am Room:CaseRm4Multi-levelclasses,wherelearnersfromdifferentyearlevelsworkingtowardsdifferentNationalCertificateof EducationalAchievement (NCEA) levels and assessments are taught togetherwithin a single languageclassroom,arebecomingincreasinglycommoninNewZealandsecondaryschools.Thisiscitedasoneofthemostsignificantchallengescurrently facing teachers.The literatureondifferentiationsuggests thatwhileteachersunderstandtheprinciples,theyfinditdifficulttoimplementthemeffectivelyintheclassroom.
Anadditionalchallengeisthatthemajorityofresearchinthisareahasfocusedonteachingandassessingclassesofmixed-abilitylearnerswithinthesameyearlevelworkingtowardsthesameassessment.Thereisagapinthefieldlookingatmixed-levelclasses,wherewithinasingleclass,inadditiontoteachinglearnersacrossarangeofabilities,teachersareexpectedtoteachdifferentcurriculumcontenttolearnersateachyearlevelinordertopreparethemfordifferentNCEAassessments.
This paper presents the findings from a teacher survey and interviews looking at how teachers preparestudents forNCEAassessments in themulti-level language classroom. It starts by summarising teachers’feelings about teaching multi-level classes, their level of confidence in teaching them, and the mainchallengesandbenefitsexperiencedbyteachersandlearners.Theremainderofthepaperfocusesonhowteachersplanforteachingmulti-levelclassesandexplorestherangeofstrategiesthatareusedtogetherwithteachers’viewsonwhichstrategiesarethemostsuccessfulinpreparinglearnersforassessment,theleastsuccessful,andwhy.
BernadetteBarker(BrisbaneCatholicEducation,Australia)Assessinglanguagethroughtasksintheclassroom:Aprocessfordeterminingwhetherastudenthas
additionalneedstolanguagelearninginEnglishDay:Saturday Time:11–11:30am Room:CaseRm1TheprogressandrateoflearningEnglishcanvarysignificantlybetweenyoungEAL/Dlearners.ThediverseexperiencesandbackgroundsoftheselearnersinfluencetheirEnglishlanguagedevelopmentandlearningoutcomes.Teachersseekingtounderstandthosestudentswhoarenotprogressingmayreferstudentsforassessmentoutsidetheclassroomcontext,usuallytoSpeechPathologistsorGuidanceOfficers.Inresponsetoreferral,standardisedtests,normedonmainstreamschoolpopulationsareusedtodeterminewhetheranEAL/Dstudenthas learningorspeechproblems.These investigationscentredonstandardisedtestingcanleadtounreliableresultswithstudentsincorrectlydiagnosedwithalearningproblem.Thispaperoutlinesaprocessof investigationusedbyBrisbaneCatholicEducation thatutilisesanactionplan for teachingandlearning in the classroom using task based assessment before consideration of referral to anotherprofessional.
SherylCooke(TheBritishCouncil,China)Selectingthegatekeepers:ThefairnessdimensionoflanguageproficiencyrequirementsforlanguageassessorsDay:Saturday Time:10:05–10:35am Room:CaseRm3Acompromiseintestfairnesscanbeconstruedasaweaknessinoveralltestvalidity(Xi,2010).Onefairness-compromisingconstruct-irrelevantfactorisraterbias,afactorthatislikelyaffectedbybiasedorinconsistentselectioncriteriaforperformance-basedlanguagetestraters.Traditionally,first-languageEnglishspeakershavebeenassignedtotheraterrole,butlarge-scaletesting,testinginavarietyoflocationsaroundtheworldoutsideoftheInnerCirclecountries,andashifttowardsreflectingarangeofWorldEnglishesinstandardisedtestinghasseenan increase in theuseof raters forwhomEnglish isnota first language.Thishas raisedquestionsaboutraterconsistencyandsuitabilityforinternationaltestsandhassparkedresearchcomparingtheratingofassessorsfromdifferentL1backgroundsandlevelsofEnglishproficiency(Hill1996;Xi&Moullan2011;Zhang&Elder2011;Harding&Griffiths2014).
Whileacertainlevelofproficiencyinthelanguagebeingassessedisclearlynecessary,howthisrequirementisreflectedintherecruitmentcriteriaforlanguageassessorshasreceivedlittleattention.Thispaperprovidesan overview of the minimum proficiency requirements for raters across a range of English languageperformance-basedtestsandconsiderstwoquestionsrelatedtotestfairness:
25
a) Arelanguageproficiencycriteriatransparentandequallyappliedtoallraterapplicantsoristherebias?b) Whataretheimplicationsoftherecruitmentpractices?
Aconsiderationofthepossibleimpactofthesepracticesonboththeraterapplicantsand,subsequently,onthetest-takersispresentedinafairnessargumentframeworkwiththeaimofencouragingscrutinyofthepotentiallyexacerbatingeffectof selectioncriteria for ratersasa contributing factor to test fairnessandtherebytestvalidity.
PaulCrump(CambridgeAssessment,UnitedKingdom)Fitnesstopractise:RevisingtheOccupationalEnglishTestListeningcomponent
Day:Saturday Time:2:15–2:45pm Room:CaseRm2TheOccupational English Test (OET) is an international English language test that assesses the languagecommunication skills of healthcare professionals looking to register and practise in an English-speakingenvironment.Itseekstoensurethatcandidatesareprepared,inlanguageterms,forworkintheirprofession.
The test is currently undergoing a revision process to ensure it is fit for purpose and reflects the latestdevelopmentsinlanguagetestingresearch.Therevisedtestisduetolaunchin2018.
ThepurposeoftheOETListeningsub-test is toprovidetestuserswithavalidandreliableassessmentofcandidates’ listening abilities in a health-related context. This presentationwill confine itself to changesplannedtotheListeningcomponentofthetestandwillfocusonthefollowingareas:
1. AnoverviewofthedevelopmentofthenewListeningspecificationandtherationalebehindit.Thetesthasbeenrevisedto includeawiderrangeofpatient:professionalandprofessional:professionalinteractionsamonghealthcareprofessionalsfromthesameanddifferentdisciplinesaswellasfromdifferentL1backgrounds.Thiswillensurethetestadequatelyreflectstherealitiesofmedicalpractice,asitisabletoincludeawiderrangeoftestingfocuses.
2. Theintroductionofscripteddialoguestothelisteningtest.The new specification includes the use of scripted dialogues, based on authentic sources, rather thanrecordingsofextemporised,semi-structuredconversations.Thepresentationwillexplorehowthenewtestaimstobalanceauthenticitywithreliability,presenttherationaleandresearchbehindthemovetoscripteddialogues,andoutlinehowauthenticitycanbemaintainedinascriptedtask.
MichelleCzajkowski(UniversityofMelbourne,Australia)Judgementsofwritingproficiencybynon-nativeandnativeEnglishspeakingteachers:
ComparingholisticandanalyticalscoringDay:Friday Time:3:40–4:10pm Room:CaseRm3InternationalEnglish languageproficiencytestshavestartedtoconsiderhowtobestreflectthestatusofEnglishasagloballanguage.However,itisstillsomewhatunusualforhigh-proficiencynon-nativespeakerstoperformintheroleofproductivelanguagerater.WhetherL1backgroundhasaneffectonhowdifferentratersapproachtheratingofL2writinghasbeenthesubjectofpreviousresearch,muchofwhichinvestigateshowtheratersdefinetheconstructofproficiencyforthemselvesthroughunguidedholisticrating.
Thisstudybuildsonpreviousresearchbyfirstfollowingthemethodologyoftheseholisticratingstudies,andbysecondlyextendingthismethodologybyaskingparticipantstoratethesamelearneressaysusingdetailedanalyticratingscales,simulatingtheroleofanoviceraterinhighstakestests.Holisticandanalyticratingswere collected from both native (n = 19) and non-native (n = 20) speaker teachers of ESL/EFL, aswerecommentsjustifyingholisticscores.
Theresultsshowthatratingbehaviourbetweenthetwogroupsshowedonlyaslightdifference ineitherscoringmethod,thoughnon-nativespeakerswereconsistentlyharsherintheirratingsandfocusedmoreonnegativeaspectswhenrating.AnalysesusingFACETSshowedarangeofratingbehaviourswithinthetwogroupswithagreatdealofoverlap.Theseresultssuggestthatnativespeakerteachersmaynotbeinnatelyadvantaged.Theyalsosuggestthatdescriptiveanalyticratingscalescanreducevariancebetweenindividualnoviceraters,regardlessofL1.
26
PeterDavidson(ZayedUniversity,UnitedArabEmirates)AssessingEAP:TheCaseforAuthenticAssessment
Day:Friday Time:2:30–3pm Room:CaseRm4AuthenticassessmentisoneofthekeyprinciplesoflanguagetestingthatisbecomingincreasinglyprominentinthefieldofEAPandlanguageassessment.InthispaperIwilloutlineexactlywhatauthenticassessmentis,contrasting itwith traditional assessment. Iwill then look at a needs analysis to determinewhat is thatuniversitystudentsactuallydointheuniversitycontext.ThisneedsanalysisinformedthedevelopmentofarangeofauthenticEAPassessmenttasksthatIwillshowcase.TheseauthenticEAPtasksconsistoflisteningandretellingalecture,writinganessaybasedonreadingsemi-academictexts,participatinginanacademicdiscussion,andcompletinganinformationliteracyproject.Thiswillbefollowedbyabriefdiscussiononhowtodealwiththeinevitableresistanceyouarelikelytogetwhenimplementingauthenticassessmentforthefirsttime.BytheendofthistalkIhopetohaveconvincedparticipantsthatauthenticassessmenthasthepotentialtoproducemoreaccurateassessmentsofawiderrangeofEAPconstructsthantraditionaltypetesting. Furthermore, because authentic assessment utilizes assessment tasks that replicate the types oftasksthatstudentswillberequiredtoactuallydointheiractualuniversitycontexts(Davidson,2009),itismore likely tohaveapositivewashbackeffect,and it isabetterpredictorofacademicsuccess, thanthetypicaltestswecurrentlyimplement.
SimonDavidson(UniversityofMelbourne,Australia)CandoctorssetvalidstandardsonanESPtestforhealthprofessionals?
Day:Saturday Time:1:05–1:35pm Room:CaseRm2Aspartoftheprerequisitetoobtainprofessionalregistrationandpractice inAustralia,overseas-qualifieddoctorsneedtodemonstratesatisfactoryEnglishlanguageproficiency.Concernshavebeenraisedthatthespecified minimum level of proficiency needed on language tests used for this purpose (including theOccupational English Test (OET), a specific-purpose language test for health professionals), might beinadequateforfunctioningsuccessfullyintheworkplace.Inanswertotheseconcerns,astudywascarriedouttosettheminimumstandardsofperformanceonthewritingsub-testoftheOETviatheprocedureof‘standardsetting’.Standardsettingisamethodicalprocessofelicitinginsightsfromrelevantstakeholdersaboutlevelsofproficiencythatareregardedasadequateforaparticularpurpose.WhilethereisaclearneedtoestablishdefensiblestandardsontheOET,thereisalackofresearchthusfaronwhatinformsjudgementsaboutsuchstandards.
Thestudysoughttoanswerthequestion:WhataspectsofOETwritingperformancedohealthprofessionalsattendtoinmakingtheirjudgementsandtowhatextentaretheylanguagebased?Toinvestigatethebasisfordoctors’perceptions,verbalreportsintheformofathinkaloudprotocol(TAP)wereutilised.Fivedoctors,allwithexperienceofworkingasGPs,specialistsandmedicaleducatorswereaskedtosay‘outloud’whatthey think or notice while reading and judging the adequacy of 10 OET writing responses selected torepresentdifferentlevelsofwritingability.Thedoctors’commentswerecodedthematicallyandintercoderreliabilitycheckswerecarriedout.
The findings showed thateachof the fivedoctors attended to similar aspects, however therewas somediscrepancybetweenwhatwasremarkedon.SomeofthefeaturesmentionedappearedtohavemoretodowithclinicalcompetencethanwiththedimensionsofcommunicativecompetencewhichtheOETisdesignedtoassess.ThishasimplicationsfortheconstructvalidityoftheOETstandardsinparticularandforstandard-settinginESPtestingmoregenerally.
27
SueEdwards(WaikatoInstituteofTechnology,NewZealand)IssuesforNewZealandprimarymainstreamteachersassessingEnglishlanguagelearners
Day:Saturday Time:11:35am–12:05pm Room:CaseRm1AsignificantproportionoftheNZprimaryschoolpopulationnowcomprisesEnglishlanguagelearners(ELLs).IfschoolswishtoapplyforadditionalfundingtosupportELLs’learning,mainstreamteachersarerequiredtoassesstheirELLstwiceyearlyandplacelearnersattheir‘achievedlevel’oftheEnglishLanguageLearningProgressions(ELLP)(2008).KeyguidelinesforteachersastheyassessELLs’languageskillsarethattheyareto “use a wide range of assessment tasks, activities and observations to make an OTJ (overall teacherjudgment)withreferencetothevariousdescriptorsontheELLPmatrices”andthattheassessment“shouldnotbeseenasadditionaltotheschool’snormalassessmentschedulebutasanintegralpartofit” (MinistryofEducation2015).
This paper describes a pilot study involving a small number of primarymainstream teachers,whowereinterviewedusingasemi-structuredinterviewschedule,andaskedtoreporttheirpracticesandperceptionsinregardtoformingtheirOTJsoflearners’Englishlevelsforfundingeligibility.SpecificquestionswerealsoaskedregardingseveralpotentialissuesformainstreamteachersastheyassessELLsforfundingeligibility.The firstof these issueswashowteachersdeterminewhat the“wide range”ofassessment tasks shouldconsist of, and which of their normal assessment practices can provide appropriate evidence for ratinglearners’skills.AsecondquestionwashoweasilyteachersareabletointerprettheELLPdescriptors,astheseinclude language-focussed terminology which mainstream teachers may not be familiar with. A thirdquestionwashowteacherstrytoensureassessmentreliability,orconsistencyoftheirOTJswiththoseofotherteachers.Teacherswerealsoaskedtoprovidetheiropinionsaboutusingthefundingeligibilitysystem,andtheirconfidenceandcompetenceinusingthesystem.
RosemaryErlam(TheUniversityofAuckland,NewZealand)Usingevaluationtopromotechangeinlanguageteacherpractice
Day:Friday Time:12:30–1pm Room:CaseRm4Recent literature in teacher education has argued for a shift away from the development of teachercognitions as a goal of teacher education to the development of core practices which would make adifferencetostudents’livesintheclassroom(Ball&Forzani2009;Kubanyiova&Feryok2015;Zeichner2012).HeibertandMorris(2012)proposethatthesekeypracticeswouldbeembeddedintoinstructionalcontextsandpreservedaslessonplansandascommonassessments.
Thispaperfocusesonevaluationtoolsdevelopedforanin-serviceprofessionaldevelopmentprogrammeforlanguageteachers(TeacherProfessionalDevelopmentLanguages(TPDL,http://www.tpdl.ac.nz)).TPDLisayear-longprogrammeforteachersofforeignlanguagesinNZschools.ProgrammeparticipantsarevisitedbyTPDLIn-Schoolsupportfacilitatorsfourtimesduringthecourseoftheyear.Thefacilitatorsobservetheirteachingpracticeandthenusetwokeydocuments,the“EvidenceofPrinciplesandStrategies(EPS)portfolio”andthe“ProgressStandards”toassistteacherstoevaluatetheirpracticeagainstkeycriteria.Astheyearprogressestheteachersareincreasinglyencouragedtotakeownershipandcontroloftheuseofthesetools,sothatbyVisit4,theevaluationisconductedasaself-assessment.Thispresentationevaluatesthesetoolsandconsidersevidencefortheirvalidity.Dataispresentedfromthecasestudyofoneteacher,tofurtherdemonstratehowthetoolsareusedandtodocumentevidenceforanychangeinteachingpractice.
JanEyre(NewZealandCouncilforEducationalResearch,NewZealand)StartingPointsListening:anonlineassessmentforbeginningEnglishlanguagelearners
Day:Friday Time:2.30–3pm Room:CaseRm3ThispaperwillexplorethedevelopmentofanonlinevocabularyassessmentforbeginningEnglishlanguagelearners.Thisassessment,StartingPointsListening,wasrecentlyreleasedintrialmodeaspartoftheAdultLiteracyandNumeracyAssessmentTool(ALNAT).
ALNATwasintroducedtothetertiarysectorinNewZealandin2010.It ispartofacoordinatedsystemofresourcesdevelopedbytheNewZealandTertiaryEducationCommissiontosupportliteracyandnumeracyskillsdevelopmentforadults.AlltheseresourcesarebasedontheLearningProgressionsforAdultLiteracyandNumeracy,whichprovideaframeworkthatshowsstagesthatlearnerstypicallymovethroughastheydevelopexpertiseinliteracyandnumeracy.
28
The purpose of the Assessment Tool is to provide robust and reliable information on the literacy andnumeracyskillsofadults,includingEnglishlanguagelearners.Thisinformationcanbeusedinvariousways,suchastoinformthedevelopmentofteachingandlearningprogrammes,asameansoftrackinglearners’progress,andtoenableorganisationstocollateandreportonlearnerprogress.
SincetheintroductionoftheTool,educatorshavecalledformoretailoredoptionstomeetthespecificneedsoftheir learners. InresponsetofeedbackfromtheESOLsector,workbeganonStartingPointsoptions in2014.Thissessionwillexplorethedevelopmentofoneoftheseoptions:StartingPointsListening.
Wewillexplore thedevelopmentof theStartingPointsListening fromthe initial consultation throughtoreleasetothesector:aperiodofalmosttwoyears.Inparticular,wewillconsiderthechallengesofdesigninganonlinelanguageassessmentforlearnerswhomaybeunfamiliarwithcomputersandwhohaveverylowlevelsofliteracyinEnglish.FaisalFaisal(PurwokertoMuhammadiyahUniversity,Indonesia&UniversityofAuckland,NewZealand)
Assessingwriting:acertifiedteacher’sperception(apreliminaryresearchfinding)Day:Saturday Time:11:35am–12.05pm Room:CaseRm4Assessmentisanimportantpartoftheteachingandlearningprocessandplaysanimportantroleinknowinglearners’problemsaswellasmeasuringtheirprogressandoutcomesafteraperiodoflearningtime.Thus,theIndonesiancurriculahavemandatedteacherstoassesstheirlearners’performanceappropriately.Forlearners,asuitablyadministeredassessmentwillencouragethemtotakepartmoreactivelyandoptimizetheirlearningimprovement.Forteachers,itwillhelpthemreflectandimprovetheirpracticeofinstructionandmeasurementwhichlatercontributetothedevelopmentofamoreeffectiveclassroommanagement.
ThispaperpresentspreliminaryresearchfindingsofacertifiedEnglishteacher’sperceptionaboutassessment,particularlyinassessingthelearners’writingskill.Thedataareobtainedfromsemi-structuredinterviewsasoneofthetechniquestocollectdatainthisqualitativecasestudyinvolvingfifteenEnglishteacherswhowererandomlyselectedandvoluntarilytookpart.Thedataareanalysedbyfollowingtheprinciplesofconversationanalysisandprimarilytorevealhowtheteacherperceivesthecurricula’snotionsofassessmentofwritingskillandwillbringthemintopractice.
JinsongFan(FudanUniversity,Australia)Factorstructureandfactorialinvarianceofauniversity-basedEnglishtest:Alongitudinalstudy
Day:Friday Time:3:05–3:35pm Room:CaseRm3Understandingthefactorstructureofalanguagetestiscrucialtotheestablishmentofitsconstructvalidity(e.g. AERA, APA,&NCME, 2014). Despite the growing number of factor structure studies in the field oflanguagetesting(e.g.Gu2014; In’nami&Koizumi2011;Sawaki,Stricker&Oranje2009),almostnoneofthemusedmulti-yeartestdatatoinvestigatethefactorstructureandfactorialinvarianceoflanguagetests(seeSims&Kunnan2016foranexception).
Adopting longitudinal design and multi-sample Confirmatory Factor Analysis (CFA) as its analyticmethodology,thisstudyinvestigatedthefactorstructureandfactorialinvarianceofahigh-stakesuniversity-basedEnglishproficiencytest.Thedataofthisstudywerestudents’section-levelscoresonthetestacrosstwoyears(2014and2015).Basedonrelevanttheoriesoflanguageabilityandpreviousresearchofthistest,fourtheoreticalmodelswerespecified,includingacorrelatedfour-factormodel,ahigher-orderfactormodel,andtwocorrelatedtwo-factormodels.Thesefourmodelswerethentestedagainstthetestdataofthetwoyearsrespectively,andcomparedwitheachothertodeterminethebest-fittingmodel.Finally,multi-sampleCFAanalysiswasperformedtoinvestigatewhetherthefinalbest-fittingmodelhadthesameconfigurationsonthetestsacrosstwoyears.
Results indicated that the higher-order factor model best fit the test data; multi-sample analysisdemonstratedthatthismodelhadbasicallythesameconfigurationsonthetestsacrosstwoyears,therebysupportingtheprincipleofmeasurementinvariance.Theresultsofthisstudyaregenerallyconsistentwithprevious factor structure studies (e.g. Sawaki, et al. 2009), suggesting the complicated and hierarchicalstructureof language ability.While lending crucial empirical support to the construct validity and score-reportingpolicyofthisEnglishtest,thisstudyalsohasmethodologicalimplicationsforothertestingagenciesintheirtestvalidationandevaluationendeavours.
29
ChristinaJudyFernandez(UniversityofSydney,Australia)Testtakers’speakingstrategies:“Ido,IthinkandIthinkaboutmythoughtsbecause…”
Day:Saturday Time:1:40–2.10pm Room:CaseRm3AlthoughmanystudieshavebeenundertakentovalidateBachmanandPalmer’s(1996)notionofstrategiccompetence,thesehavemostlyfocusedonreading,listeningandwritingtestsandnotmanyonthespeakingtestdomain.Itisthisgapinknowledgethatthisqualitativestudyfillsandcontributesto.Thispresentationreportsonastudyexploringtesttakers’strategyuseinatwo-waydiscussion(orPart3)oftheInternationalEnglishLanguageTestingSystem(IELTS)speakingtest.Thestudysoughtanswerstothefollowingquestions:1. Whatcognitive,metacognitiveandcommunicationstrategiesdotesttakersemploytoenhancetheir
testperformance?2. Howdotesttakersperceivetheusefulnessoftheircognitive,metacognitiveandcommunicative
strategyusetoaddressthetwo-wayIELTSdiscussion?3. Whatareindividualandcontextualfactorsaffectingtheiruseofstrategiesduringthetesttask?
Datawere collected from12 international students in Sydney,Australia. Theparticipants consented toa5-minutetwo-waydiscussionwhichwasvideorecorded.Thiswasfollowedbyastimulatedrecallsessionoftheirspeakingtestexperience.Thestimulatedrecallsweretranscribedandcoded.
Detailedanalysisofstrategiesusedbyafewparticipantswillbeprovidedanddiscussedinthepresentation.Somekeyfindingsandimplicationsofthisstudyonthetwo-waydiscussionoftheIELTSspeakingtestwillalsobehighlighted.
KellieFrost,UteKnoch&AnnemiekHuisman(UniversityofMelbourne,Australia)Settingstandardsonapost-entrylanguageassessment:Exploringdifferencesinvaluesofcontent
lecturersandacademicskillsstaffDay:Friday Time:11:20–11:50am Room:OGGB5Linguistically diverse student populations within Australian tertiary institutions have meant that a largenumberof tertiary educationproviders nowassess the English language ability of all or certain targetedstudentgroupspost-entry.ThisisdesignedtoensurethatstudentswhoarelikelytoexperiencedifficultiesareidentifiedearlyandguidedtowardsEnglishlanguagesupportopportunitiesavailableoncampus.Whileseveral research studies and books have focussed on issues surrounding these post-entry languageassessments(PELAs)(seee.g.Read2014;Knoch&Elder2013forsummariesofsuchresearch),itisnotalwaysclearhowstudentstakingsuchassessmentsarecategorisedintermsofrequiringsupport.Furthermore,thedocumentationontheseassessmentsrarelyspecifieswhosestandardsofEnglishlanguageproficiencyareusedtocategorisestudents,contentlecturersortheacademicskillsstafftypicallyresponsibleforprovidinglanguagesupportacrosscampuses.
ThispaperreportsonaprojectaimedatsettingmeaningfulstandardsontheDiagnosticEnglishLanguageAssessment (DELA), a PELA used at the University ofMelbourne since the early 1990s. The projectwaspromptedbyarecentlyproposedpolicychangetobroadenthestudentbasethatwouldbeassessedpost-entry,which sparked discussions about standards on the assessment. To ensure the standards used arecurrentandsetempirically,standard-settingworkshopswereconvenedtoestablishtheminimumrequiredstandardsofEnglishproficiencyexpectedby:(1) Contentlecturersfromarangeofdisciplines,and(2) Academicskillsadvisorystaff.
Standardsweresetbybothgroupsforstudentsrequiringcompulsorysupport,recommendedsupportandnolanguagesupport.Theresultsshowthatthestandardsareremarkablysimilaracrossthetwogroupsofstakeholders.Aqualitativeanalysisof the reasonsgiven for judgementsduring theworkshops,however,showed somedifferencesbetween theaspectsof languagevalued. The findings arediscussed in lightofstandard-settinginacademiccontexts.
30
CateGribble(DeakinUniversity,Australia)Englishlanguageproficiencyandemployment,migrationandprofessionalregistrationoutcomesin
healthcareandearlychildcareeducationinAustraliaDay:Saturday Time:1:40–2:10pm Room:CaseRm2This presentation reports on a British Council, Cambridge English Language Assessment, and IDP: IELTSfundedprojectthat investigatedtheroleof IELTS inshapingthesupplyofskilled labouranddeterminingemploymentoutcomesofinternationalgraduatesandoverseastrainedprofessionalsinhealthcareandearlychildhoodprofessionsinAustralia.
Focusing on two professional fields that are critical to Australia’s future – health and early childhoodeducation–thestudyprovideskeyinsightsintothemanychallengesfacingoverseas-trainedprofessionalsandinternationalgraduatestransitioningintotheAustralianlabourmarket.
The study incorporated qualitative research methods to investigate the role of IELTS in determiningemployment and migration outcomes of overseas-trained graduates in health and early childhood.Interviews (n = 77) with professional bodies, employers, academics, overseas-trained graduates andAustralianinternationalgraduateswereconductedinMelbourneandtworegionalsites.
Theresultsofthestudyhighlightthecomplexlanguagerequirementsintheprofessionsofmedicine,nursingandearlychildhoodeducationandrevealkeyimplicationsforstakeholders.Employersrequiregraduatestohave high-level English language skills, and universities are increasingly expected to ensure internationalstudentsgraduatewiththerequiredEnglishlanguageproficiency.Thestudyalsorevealshighlydifferentiatedlabourmarkets.Whilemetropolitanhospitalsareturningawaybothdomesticandinternationalgraduates,someruralhospitalsarepredominantlystaffedbyinternationaldoctorsandnurses.Earlychildhoodeducationalsofacesskillshortagesinruralareasandsomeareasarestrugglingwithhowtostemtheflowofgraduatesintotheprimarysector.
Finally,thestudyhighlightsthemanychallengesfacingoverseas-trained/internationalgraduatestransitioningintothelabourmarket.Forparticipants,thechallengesofworkingintheirprofessioninAustraliaaremanyand varied. These challenges include workplace discrimination, isolation and extreme frustration whenunabletoworkintheirareaofqualification.
PeterGu(VictoriaUniversityofWellington,NewZealand)CreatingandvalidatingtheClassroomAssessmentConfidenceIndexamongChineseEFLteachers
Day:Friday Time:3:05–3:35pm Room:CaseRm4ThispresentationoutlinestheprocessincreatingandvalidatingtheClassroomAssessmentConfidenceIndex(CACI) for EFL teachers. We will first present a framework for conceptualising ‘classroom assessmentcompetence’,followedbydescriptionsofthedesignandvalidationprocess.
Theconceptualframeworkwasdevelopedbasedonacomprehensivereviewoftheoriesandresearchonassessmentliteracyandclassroomassessment,fromclassicconceptualisationsofassessmentliteracysuchastheStandardsforteachercompetenceineducationalassessmentofstudentsbytheAmericanFederationof Teachers, theNational Council onMeasurement in Education and theNational EducationAssociation(1990)andStiggins(1995),tomorerecentreformulationssuchasHeritage(2007),Brookhart(2011),Willis,Adie,andKlenowski(2013),andDeLuca,LaPointe-McEwan,andLuhanga(2015).Theframeworkalsorefersto latest applied linguistics research on classroom assessment practices (e.g. Davison & Leung 2009;Rea-Dickins2001;Rea-Dickins&Gardner2000;Taylor2009).
A 40-item Likert-scale CACIwas created based upon the conceptual framework and upon an analysis ofexistingassessmentliteracymeasures(Gotch&French2014),aimingtoelicitEFLteachers’self-assessmentof their own confidence in classroom assessment practices. One hundred and twenty secondary schoolteachersteachingEnglishasaforeign language inChinaparticipated inthevalidationsurvey.Exploratoryfactoranalysisrevealed8factors:1) Believinginclassroomassessment,2) Havingclearstandardsbeforeassessment,3) Abilitytochooseanddesignassessmenttools,4) Abilitytodoflexibleonlinemonitoring,
31
5) Abilitytoevaluateassessmentresultsanddiagnoseproblems,6) Abilitytoadjustteachingandlearning,7) Abilitytoreporttostakeholders,and8) Abilityinethicalassessment.
Confirmatoryfactoranalysiswasnextperformedtoobtainmodel-fitstatisticsforthe8-factormodel.DiscussionwillfocusontheuseandlimitationsofameasuresuchastheCACIforteachereducationprogrammes.
KarenHuang(UniversityofAuckland,NewZealand)DevelopingaChineseplacementtestforheritagestudentsintertiaryeducation:Issuesandconcerns
Day:Friday Time:10:20–10:50am Room:CaseRm2AnincreasingnumberofstudentswhospeakChineseathomeintendtostudyChineseintertiaryinstitutions.However, theseChineseHeritage Language Learners (CHLLs) showawide rangeofMandarinproficiencylevels.SomespeakMandarinfluentlybutcannotreadorwrite;somecanunderstandbutrarelyspeakthelanguage;somecanread/writeChinesecharacters,butdonotspeakanyMandarin—theyspeakCantonese.Although it is extensively accepted that CHLLs need to receive separate instructions, the budgetaryconstraintsanddiversestudentprofilesmakeitimpossiblefortheprogrammetoaccommodatealloftheCHLLs.Chineseprogrammesareforcedtoeitherplacestudentsinoneoftheforeignlanguageclassrooms,orturnthemdownfromstudyingChinese.
ThisstudyaimstoanalysetheissuesanddifficultiesincurrentplacementpracticesandproposeaframeworkforCHLLplacement.Thecurrentpracticesincludeabiographicbackgroundquestionnaire,anoralinterviewandawrittentestthatmightbeanessayorreadingcomprehensionquestions.However,thesetestsoftenfail to capture the true proficiency of these CHLLs. Their grammar knowledge might be overestimatedbecauseoftheirfluencyandthetopicthathappenedtobeasked;whiletheymightbeunderestimateddueto their lower reading and writing ability. This study identified features in CHLLs’ phonology andmorphosyntax,andfurtherdesignedaseriesofquestionsthatassessCHLLs’grammaticalknowledgesuchastheuseofclassifiers,relativeclauses,complexwordorder,grammaticalaspects,passiveandthedisposalconstruction,aswellassimplequestionsthattesttheirknowledgeonliterarystratumandvocabularyranges.Bydevelopinganobjectivemeasurement,thisstudyintendstofindamorepracticalplacementsolution.
PamelaHumphreys(GriffithUniversity,Australia)TheoreticalandconceptualmodelsofacademicEnglishlanguageproficiencyinhighereducation:
ConsiderationsforprincipledassessmentinEAPDay:Friday Time:10:20–10:50am Room:OGGB5ModelsofcommunicativecompetenceandEnglishlanguageproficiencyarewellestablishedinthefieldsofTESOLandAppliedLinguistics(e.g.Bachman1990;Canale1984;Canale&Swain1980;Celce-Murcia2007;Celce-Murcia&Dornyei 1995; Purpura 2004). But, given thehighnumberof international studentswithEnglishasanadditionallanguage(EAL)preparingforandundertakingdegreestudiesinouruniversities,howwelldoweunderstandtheconstructofacademicEnglishlanguageproficiency?Asakeystoneattributeforacademicsuccess(Humphreys2015;Sawir,etal.2012;Woodrow2006),itisarguedthatweneedtobetterunderstandthisconstruct.
ThispresentationwillprovideanoverviewoftheextantconceptualframeworksrelatedtoacademicEnglishlanguageproficiencyinthehighereducationcontext(Arkoudis&O’Loughlin2012;Harper,Prentice&Wilson2011;Mahboob2014;Murray2010).AheuristicofEnglishLanguageProficiencyforthiscontextwillalsobeproposed (Humphreys 2015), which combines the aforementioned frameworks in a new way to aidconceptualisation.ThissessionwillbeofparticularinteresttothoseinterestedintheimplicationsofsuchframeworksforprincipledassessmentinEAP.
32
NaokiIkeda(UniversityofMelbourne,Australia)AssessingL2learners’oralpragmaticandinteractionalabilitiesforuniversitysettings:Implicationsfor
classroomassessmentDay:Friday Time:12:30–1pm Room:OGGB5Pragmaticand interactional competenceareunderrepresented in theconstructof large-scaleproficiencytestsforuniversityadmission(Roever2011).Furthermore,theassessmentofpragmaticand interactionalcompetence is rarely undertaken in L2 classrooms (Youn2013). Therefore, limited informationhas beenprovided aboutwhat features of pragmatics are challenging for L2 students, how these features can bedifferentiatedand,whatspecificfeaturescouldbetargetedsothatstudentscansuccessfullyparticipateinuniversityactivities.
The present study, by integratingmultiple sources of data, aims to identifymeasurable features of oralpragmaticsandinteractiontoconstructinstrumentsfortask-basedclassroomassessmentsoforalpragmaticabilitiesforuniversityactivities.
Oraldiscoursedatawerecollectedfrom67L2studentsinAustralia(currentuniversitystudentswithIELTSequivalent6.0to8.5andpre-entrystudentswithIELTS6.0to6.5)whocompletedthreedialogueandthreemonologue role-play tasks simulating university situations. The data also include their self-assessed taskperformances,interlocutors’perspectivesandraters’judgments.Thedatawereanalysedbothquantitativelyandqualitatively throughuseof a rangeofmethods: descriptive statistics, discourse-based analyses andRaschanalyses.
Thefindingssuggest thatthere isacleardifferencebetweenL2students’performances intermsof theirdiversityoflinguisticresourcestohandlepragmaticdemands(Bardovi-Harling2013)andabilitiestotailortheirsocialactions(Kasper2006)forthecontext.Interactionalfeatures(e.g.repair,turn-taking)uniquetosomepre-entrystudentswerealsoidentified.Inaddition,Raschanalyseshaverevealedtowhatextentandhowdemandingeachfeatureisforthestudents.
Bydrawingon interviewdatathat investigates learners’perceptionsaboutthedesigned instrumentsandtheirtaskperformances,thisstudywilldiscussthemeasurableconstructsofpragmaticsandinteractionwithrelevancetoboththedesignandimplementationofclassroomassessmentinstrumentsinordertobetterassessL2learners’readinessforuniversity.
LeilaIranmanesh(UniversityofNewSouthWales,Australia)EvolutionofformativeassessmentinanEnglishacademicwritingclass:Theroleofemotionandpower
Day:Friday Time:11:55am–12:25pm Room:OGGB5Lookingthroughthelensofcriticalemotiontheoriesinthispresentation,Iwilldiscussapartofanin-depthinquiry intothe integrationandevolutionofformativeassessment inanEnglishacademicwritingclass inIran.Accordingly,Iwillalsoexploreinwhatwaysformativeassessmentevolvedandimpactedteachingandassessment practices as well as participants’ understanding of assessment practices in a critical actionresearch.Thedatacomprise thesystematicobservations, interviews, reflections,detailed fieldnotesandsampleofparticipants’writings.Thecyclesofplannedactions,reflectiveevaluationandrevisedactionsshedlightondifferentdimensionsofevolvedformativeassessment,shiftsinparticipants’emotions,learningandunderstanding and the challenges encountered by 9 participants; learners and I as the teacher andresearcher.Takingtheparticipants'voicesandemotionsintoaccountfromcriticalperspectives,Iwilldiscussthe implications forpolicy,programdevelopment,academicwritingpedagogyand future research in theassessmentcontext.
PeterKeegan(UniversityofAuckland,NewZealand)TriallingaMāorilanguagepronunciationtoolbasedonaMāorispeakerdatabase
Day:Friday Time:10:20–10:50am Room:CaseRm4TheMAONZEproject(MaoriandNewZealandEnglish)usesrecordingsfromthreesetsofspeakerstotrackchangesinthepronunciationofMāori(theindigenouslanguageofNewZealand)andevaluateinfluencesfromEnglish.Thefirstgroupofspeakerswereborninthelatenineteenthcenturyandrecordedmostlyin1946-48.Thesecondgroupofspeakersarekaumātua/kuia(elders)bornbetween1920and1940,andthethirdgroupareyoungspeakersbornbetween1970and1990.
33
Resultsfromtheprojectshowchangesinbothvowelqualityandvowelduration(forallagegroupsandbothgenders) andevidenceof diphthongmergers especially amongst the younger speakers. Female speakersfromallthreeagegroupswereaheadofthemalespeakersintermsofraisingtheshortvowels/e/and/o/andinglideweakeninginthediphthongs.Theyoungwomenarealsoinadvanceoftheyoungmenin/u/fronting.
Inthispresentationwedescribetheresultsofdevelopingandtriallingseveralphasesofacomputer-basedaidthatassists learnersto improvetheirownpronunciationofMāori.Severalprototypesoftheaidhavebeendeveloped.The first trial (n=11) tookplace in late2015, thesecondtrial (n=35)onan improvedversion,tookplaceinthefirsthalfof2016.Thosetriallingtheaidwerepositive,founditeasytouse,andthatitprovidedinformativefeedback.Theaidallowsuserstogetrealtimefeedbackontheirownpronunciationofindividualvowels,diphthongsandcommonlymispronouncedMāoriwords.Italsoallowsuserstolistentoandcompare theirpronunciationswith ‘gold standard’pronunciationsofkaumātua (eldermales)orkuia(elderfemales)bydrawingonthespeakerdatabasedevelopedbytheMAOZNEproject.Weconcludewiththoughtsonpossiblefuturedirectionsfortheaid.
JeanetteKing(UniversityofCanterbury,NewZealand)TuhingaMāhorahora:acorpusofchildren’swritinginMāori
Day:Friday Time:11.20–11.50am Room:CaseRm4Each week, children in Māori medium classrooms are encouraged to write in Māori in their TuhituhiMāhorahorawritingbook.Theaimofthiswritingprogrammeisto“helpchildrendeveloptheirownpersonalwritingvoice”(MinistryofEducation2008,p5).TheTuhituhiMāhorahoraprogrammealsoprovidesteacherswith an opportunity to assess their students’ productive output in Māori and devise interventions andstrategiestosupportthewritingdevelopmentoftheirstudents.
The67,168wordTuhingaMāhorahoracorpuswasdesignedandcompiledtotrialanalyseswhichcanassistteachersintheirlanguageenrichmentstrategies.Thecorpuscomprises1,329piecesofwritingcollectedin2013from69year1–8childrenataMāori-mediumschoolinChristchurch.Thechildren’swritinghasbeentranscribed andmarked up in oXygen and entered into LaBB-CAT, a browser-based searchable linguisticanalysistool(Fromont&Hay2012).WeusePaulNation’s(http://www.victoria.ac.nz/lals/about/staff/paul-nation)RangeprogramandWordSmithconcordancetoolstoconductsomeoftheanalyses.
Herewereportontheresultsoftheanalyseswhichrevealthevocabularythatthe5–13yearoldstudentsareusing,and,crucially,thevocabularynotyetwithintheproductivelanguageoutputofthelearner.Wealsodemonstratehowthisinformationcanbeusedtoinformteachingpractice.
At presentwe know too little about how children are using te reoMāori in immersion classrooms. TheTuhingaMāhorahoraprojectcanaddtoourunderstandingofwhatchildren’simmediatevocabularylearningneedsare,andcanhelpensurethatprogrammesprovidebothquantityandqualityofexposureinthetargetlanguage,thusenablingchildrentoexpressthemselvesadequately.
LinLin(UniversityofHongKong,HongKong)Investigatingtherelationshipsbetweensecondlanguagetesttakers’strategyuseandChinesereading
comprehensiontestperformanceDay:Friday Time:11:55am–12:25pm Room:CaseRm2The important roles of metacognitive and cognitive strategies in language test performance have beenrecognized in the theoretical models (Bachman 1990; Bachman & Palmer 1996) and empirical studies(Purpura1999;Phakiti2003,2008;Zhang&Zhang2013;Zhang,Goh&Kunnan2014).However,thereisrareconsensus on the relationships between second language (L2) test takers’ metacognitive and cognitivestrategyuseandtheirtestperformance.Inaddition,comparedwithmetacognitiveandcognitivestrategyuse, the nature of affective strategy use, another essential type of strategy discussed in the previousliterature,anditsrelationshipswithL2testperformancearepoorlyunderstood.
The current study examines L2 test takers’ strategy use through a questionnaire and the relationshipsbetweentheirstrategyuseandtheirChinesereadingtestperformanceonalarge-scalestandardisedChineseproficiency test: Hanyu Shuiping Kaoshi (HSK). It particularly investigates the nature of metacognitive
34
(planning, evaluating, and monitoring), cognitive (comprehending, memory, and retrieval) and affective(activating supportive emotions, and generating and maintaining motivation) strategy use and theirrelationshipswithreadingtestperformanceassessedbyliteralandinferentialcomprehensionquestions.Thestudyinvolves562L2intermediate-leveltesttakerswhostudiedChineselanguageatuniversityinmainlandChina.Thetesttakersreportedona56-itemstrategyusequestionnaireimmediatelyaftertheycompleteda45-itemHSKreadingsubtest.
Results showed metacognitive strategy use and affective strategy use were significantly correlated andmetacognitive strategy use had a significant effect on cognitive strategy use in the test context.Comprehendingandretrievalstrategieshadpositiveeffectsonliteralcomprehension.Itwasalsofoundthatliteralcomprehensionhadasignificanteffectoninferentialcomprehensioninthereadingtest.FindingsfromthestudyprovidepracticalimplicationsforlanguageteacherstoteachreadingcomprehensionstrategiesinclassroomandforHSKtestdesignerstodesignreadingtestitems.
XiaohuaLiu(UniversityofAuckland,NewZealand)Analysingexistingreadingtesttasks:Implicationsfordevelopingtaskstomeasuredifferentreadingabilities
–developmentandvalidationofdiagnosticlanguageassessmenttasksDay:Friday Time:2:30–3pm Room:OGGB5The recent growing interest in diagnostic language assessment has brought about a number of studiesattemptingtoretrofittheinterpretationoftestresultsofexistingreadingtestsdevelopedforotherpurposes(e.g.selection,placement),throughanalysingreadingtasksandtesttakers’performances.Findingsofthesestudiesshowapromisingpicture:althoughafewstudiesdidnotfindclearevidenceforthedivisibilityofageneral reading construct, most of them identifiedmultiple reading subskills underlying their test data.Nevertheless,thesubskillsfoundvaryfromstudytostudyintermsofnumberandnature,andsomeofthemreflectmoreofataskprocessingmodelthanareadingmodel.Moreover,mostofthemstoppedatidentifyingsubskillsinexistingtasks,withoutexploringthefactorsthatmayfacilitateorinhibittheirassessment.
By summarising those subskills previously identified and comparing and contrasting them with readingtheories,aswellasbydrawingon findings fromstudies investigating the impactof test task featuresonreading testperformance, Idevelopeda framework (consistingof readingsubskillsandtask features) forsubjective analysis of reading tasks. Using this framework, eight language experts were invited toindependentlyanalysea setof reading tasks fromapost-entryEnglish language test administeredbyanEnglish-medium university, meanwhile verbalising their thoughts during the process. The results werecomparedwiththeverbalreportsofagroupofstudentsdoingthosetasks.Findingsoftheseprocedureswillbepresentedandtheirimplicationsfordevelopingtaskstomeasuredifferenttypesofreadingabilitywillbediscussed.
LaurieLu(NelsonMarlboroughInstituteofTechnology,NewZealand)ComingtogripswithtechnicalissuesindevelopingandimplementingEAPUnitstandardsassessments
Day:Saturday Time:10:05–10:35am Room:CaseRm1ThispaperexaminessomeofthekeytechnicalissuesthatareeitherintrinsicallytruetoEAPunitstandardsassessments or possibly acquired “iatrogenically” in the process of developing and implementing thesestandard-basedassessments.ItfocusesitsdiscussionsonchallengesimposedbytheEnglishLanguageUnitStandards,fairnessandconsistencyissues,sufficiencyissuesreachievementevidenceandtaskquantityandtypes,difficultylevelofassessmenttasks,assessmentdesigningandthesummativeandformativeuseoftheassessments.Allthechallengingfeaturesareexploredinthecontextofensuringassessmentstobefair,validand reliable. Overall, the research seeks a better understanding of these teacher-made and internallyadministeredassessments,makes senseof certain challenging featuresandattemptsan identificationofsomefeasiblesolutionsfortheimprovementofsuchanassessmentsystem.
35
SusyMacqueen(TheAustralianNationalUniversity,Australia)Profession-specificlanguagestandards:PerspectivesfromprofessionalbodiesontheuseoflanguagetestsDay:Friday Time:3:40–4:10pm Room:CaseRm4Standardsarephysicalorbehaviouralmechanismswhichmakesocietyorderly(Busch2011;Lampland&Star2009).Theymayariseorganicallyinsocietiesand/ortheymaybeimposedsothatorderisforced.Languageitselfisakindofstandard,asocialorganiserparexcellence;itevolvesorganicallythroughsocialinteractionbutitisalsoimposedviapolicyandsourcesofpower.
Inthispaper,weexplorethenotionoflanguagetestsasimposedstandards,whichareusedtobringordertoanaspectofsociety,namelyhumanmigration.Inordertobetterunderstandhowstandardisedtestsareused as filtering mechanisms for skilled migration, we interviewed those who implement the standard:membersofthesixprofessionalregistrationbodiesfortheAustralianaccounting,engineering,medicalandnursingprofessions(11interviewees).
Athematicanalysisoftheinterviewdataoffersinsightsintohowprofession-specificlanguagestandardsareusedandunderstoodintheAustraliancontext.Inparticular,findingsshowthatthemeaningsofstandardisedtestscoresareconstructedinrelationtootherstandardindicatorssuchasapplicants’professionalnarrativesandprofessionalexaminationprocesses.Languageteststandardsaretrustedmeasures(toacertaindegree),whichoperatedifferentlyfordifferentprofessions;ateststandardmightactasafirstfilterintheregistrationprocess for one profession, but a later stage check for another. In general, the board representativesperceived that the implementation of a language test standard involves responsibility to different socialworlds: test-takers,employers, thepublicand theAustraliangovernment.Wediscuss the implicationsoftheseresponsibilitiesinrelationtothoseofthetestproviders,andmoregenerally,inrelationtoassessmentliteracy.
MatthewsMMakgamatha(HumanSciencesResearchCouncil,SouthAfrica)&KathleenHeugh(UniversityofSouthAustralia)
Multilingualassessment:OpportunitiesforteacherdevelopmentandequitablelearningDay:Saturday Time:10:05–10:35am Room:CaseRm2This paper draws attention to the potential for system-widemultilingual assessment to have a positivewashbackeffectonteacherdevelopmentandenrichedteachingandlearninginclassroomswithlinguisticallydiversestudents.Thepaperisbasedonafirstsystem-widemultilingualassessmentofstudentsattendingtheWesternCapeDepartmentofEducationschoolsinSouthAfrica.
Thisstudywasconductedin2006bytheHumanSciencesResearchCouncil(HSRC).Thetestsadministeredtostudentscomprisedthreeversionsofthelanguagesinstruments(Afrikaans,EnglishandisiXhosa)andtwolanguage versions of mathematics (in Afrikaans and English). The format of items in the tests includedselectedresponsequestions(SRQs)andconstructedresponsequestions(CRQs)thatwerecraftedtoprovidediagnosticinformation.Althoughweusedonlytwolanguageversionsofthemathematicstests,theyincludedtrilingualglossesfor40%oftheitemsineachtest.Thisenabledstudentstodrawontheirtranslanguagingexpertise when answering these items. Teachers were contracted to undertake diagnostic marking andgradingofextendedresponseitems.
In this paper, our attention is on the benefits which multilingual assessment may have for teacherdevelopmentandenrichedclassroomlearning.Weprovideadetailedaccountofasystemwedesignedformarkingofthemathematicsandlanguagetestsadministered.Thepaperwilladdressthefollowingissues:(a) trainingof teachers inpreparation formarkingof tests, (b)managementof themarkingprocesswithspecial reference to marking learner responses to CRQs, (c) the process put in place to ensure qualityassuranceduringmarking, (d)gatheringofqualitativediagnostic information from learners’ responses toCRQsforthepurposeofinformingclassroomteachingandlearning.
Thepaperconcludesbyhighlightingandreflectingonthechallengesandopportunitiesrelatedtoderivingvalue (and relevance) from qualitative information obtained through diagnostic marking and grading ofmultilingualassessment.
36
MichaelMersiades(UniversityofQueensland,Australia)ValidityofEAPreadingtestinferenceitems:Apilotstudy
Day:Friday Time:3:05–3:35pm Room:OGGB5ThispaperpresentsapilotstudyintothevalidityofinferenceitemsinEAPreadingtests.Thereisnotmuchinthelanguagetestingliteratureaboutinferenceitems,buttestdeveloperscommonlyensurefairnessandconstructvalidity in inference itemsusinga frameworkbasedonChikalanga’s (1992)distinctionbetweenpropositionalandpragmaticinferences.Inthisframework,pragmaticinferencesareexcludedfromEAPtestsbecausetheyrequiretesttakerstodrawonconstruct-irrelevantexternalknowledge(Khalifa&Weir2009;Taylor2014).However,ithasbeenacknowledgedthatinpracticeitisnotalwayseasytodistinguishbetweenpragmaticandpropositionalinferences(Hughes2003;Urquhart&Weir1998).
Astudyofthevalidityofinferenceitemsrequiresatoolforreliablyanalysinginferenceitems,sothispaperasks if it is possible to elaborate on Chikalanga’s distinction to produce a procedure that enables testdevelopers to reliably distinguish between propositional and pragmatic inference items. A procedure isproposedthatencouragestestdeveloperstosystematicallyidentifytheinformationneededforatesttakerto arrive at the correct response on a particular item. These pieces of information are then classifiedaccordingtohowthetesttakeraccessesthem(bydecodingthetext,ordrawingontopicalknowledge,forexample).After theseclassifications, the itemcanbe identifiedasapropositionalorpragmatic inferenceitem.
Twoexpertratersapplytheprocedureto160testitemsacrosstwoIELTSandTOEFLpasttestpapersinordertoclassifytheitemsaspropositionalinference,pragmaticinference,ornon-inferenceitems.Theresultsgiveaninitialindicationoftheutilityofthisinferenceitemidentificationprocedureasatooltoassessthevalidityofinferenceitems,andalsoprovideasnapshotofthetypesofinferenceitemsthatappearinIELTSandTOEFLtestpapers.
AnnMoir-Scott(UniversityofAuckland,NewZealand)LocatingtheLearning:MeasuringtheimpactofL1reflectiononL2development
Day:Friday Time:11:55–12:25pm Room:CaseRm4ThispaperdetailsavarietyofinteractionistandsocioculturaltoolsusedtomeasuretheimpactofL1writtenreflectiononL2development.TheywereemployedinDoctorofEducationresearchtocollectandanalysedata inan interventionstudyconducted in fiveFrenchForeignLanguageclassrooms in fourNewZealandco-educationalsecondaryschools.TheYear11participants (n=71)werecommencingtheir thirdyearofFrenchandtheirfirstofthreeyearsoftheNationalCertificateofEducationalAchievement(NCEA).Inthishigh-stakesassessment,ForeignLanguage(FL)writingskillsareshowcasedininternally-assessedportfolios.
The data for the research came from the first and second drafts of two tasks (four drafts per student)producedduringtwothree-weekprocesswritingunitsthreemonthsapart.Collectingdatafromtwowritingunitsenabledacounter-balanceresearchdesign.Thefirstdraftsinbothwritingunitsactedaspre-testsinthisreflectioninterventionwhiletheseconddraftsactedaspost-tests.
Themeasuringtoolsdetailedinthispresentationinclude:• TheNCEALevel1Frenchassessmentscheduleusedtogatherbaselinedata.• East’s(2008)AnalyticScoringRubric,CAFanalysis,ErrorAnalysisandAljaafrehandLantolf’s(1994)
RegulatoryScaleusedtomeasureandanalyseL2development.• Oxford's(2011)StrategicSelf-regulatedModelofL2Learning(S2R),BiggsandCollis’s(1982)SOLO
Taxonomy,andCodingsupportedbyMicrosoft(2013)OneNote,Microsoft(2010)WordCountandWientjes&Hakuta’s(2010)WordsifttoanalysetheL1reflectiondata.
Abriefdiscussionofthefindingsandusefulnessofthetoolsisincludedinthispresentation.
37
PaulMoore(UniversityofQueensland,Australia)Cohesioninorallanguagetestperformance
Day:Friday Time:11:20–11:55am Room:CaseRm3Cohesion, alongsideother aspectsofdiscourse competence, is a constructwhich is variablydefined,butwhichiscommonplaceinlanguagetestscoringrubrics.Drawingondatafromalargerstudyintodiscoursecompetenceinspokenlanguagetestperformance,thispaperfocusesontheroleofcohesionindistinguishingcandidates’performanceacrosstasksandlevelsontheAptisTest.
Afterreviewinginterpretationsofthetheoreticalconstructofcohesionandhowtheseareoperationalizedinlanguagetestingresearch,wereviewrecentresearchintotheroleofcohesioninlanguageperformanceon different tasks and at different levels of proficiency.We then report on quantitative and qualitativeanalysesofaspectsofcohesionidentifiedin83test-takerperformancesonfourtasksacrossthesixlevelsoftheAptisSpeakingTest.ThemethodofdataanalysisemployeddiscoursemeasuresusedbyIwashitaandVasquez(2015)intheiranalysisofdiscoursecompetenceinIELTSSpeakingTask2performancesincludingconjunction, reference and lexical cohesion. We operationalised discourse competence in terms of thetextualfeaturesofcohesionandcoherence.QuantitativeanalysiswasperformedusingthecomputationaltoolCoh-Metrix.
AswithotherrecentresearchintoL2speaking,measuresofcohesioninthisstudywerenot,onthewhole,clearlydistinguishableinperformancesacrosstasksandlevels.Thepresentationconcludeswithadiscussionoftheutilityofcurrentmeasuresofcohesion,whetherthesemaybeadjustedtoprovideamorenuancedunderstandingoftheroleofcohesion,orwhethercohesionasanindicatoroftest-takerperformanceisbestunderstoodinconjunctionwithotheraspectsofdiscoursecompetence.
KeikoNakao(UniversityofSouthAustralia)
ReflectiveassessmenttasksforinterculturallanguagelearninginabeginnerforeignlanguagecourseDay:Saturday Time:1:05–1:35pm Room:CaseRm1Since intercultural language learning (ILL)hasgainedgreateremphasis inconceptualisingabout languageteachingandlearning,foreignlanguageprogramshaveimplementedlearningandassessmenttasksfocusingonelicitingtheintercultural.However,manyteachershavefounddifficultyinimplementingtheassessmentofILL,becauseit’snotsimplyacaseofassessinglearners’ linguisticproficiencyorknowledgeoflanguageandculture,butitalsoinvolveslearners’interculturalcapability,understanding,reflectivityandvalues,etc.(Liddicoat&Scarino2013).
Firstly,thispaperwilldiscusstwoassessmenttasksdesignedtoelicitinterculturalcapabilitiesinabeginnerJapanese languagecourseatuniversity level.These taskswere reflectiveassessment tasks,oneofwhichinvolvedparticipationinanonlinediscussionboardandtheotherareflectionpaperonstudents’interculturalexperience,languagelearningandthetasks.
The two tasks were developed based on the frameworks and pedagogical models of ILL (Byram 1997;Liddicoat2008;Tomita2013).Thepaperwillthenpresentacontentanalysisoftheresponsesof40studentstothereflectionpaper.Finally,thepaperwilldiscussthefindingsfocusedonhowstudentsreflectontheirintercultural understanding and their language and culture learning. These findings will contribute tounderstandingfeaturesofILL,anddevelopmentofpedagogy,andassessmentofILL.
DePhung(UniversityofNewSouthWales,Australia)WhatdidEAL/Dteachersactuallythinkanddowhenmarkingoralperformances?
Day:Saturday Time:10:05–10.35am Room:CaseRm4AgreatdealofrecentresearchattentionhasbeendrawntoexaminingandimprovingvalidityandreliabilityofEnglishlanguageteacherassessmentintheAustralianmainstreamschoolingsystem.However,littlehasbeen done to enhance the trustworthiness of teacher assessment of English as a second or additionallanguageordialect(EAL/D).Thispaperreportsonfindingsfromaresearchstudyaimedat:(1) ExaminingtowhatextentEAL/Dteachers’oralassessmentsareconsistent,and(2) Exploringfactorsinfluencingtheirassessments.
38
Thisstudyadaptedmaterialsdevelopedbyalargerprojectbuildingtoolstoenhanceassessmentliteracyforteachers of English as an additional language (TEAL) in Victoria into the context of EAL/D instruction inNewSouthWales.TwelveEAL/Dspecialistswereinvitedtofirstmarkthreestudents’sampleworksandthenbefollowedup.Findingsrevealedthatteachersweresignificantlydifferentfromeachotherinperceptionofstudentperformancesand in judgmentdecisionsandthattheirassessmentsweredrivenbyanumberoffactorsrelatedtoteachersthemselves,studentsandtasks.Fromthesefindings,educationalimplicationswillalsobediscussed.
ShahrzadSaif,ZahraMahdavi(UniversitéLaval,Canada)LanguageneedsofinternationalgraduatestudentsworkingasTeachingAssistants(ITAs)inCanadian
Francophoneuniversities:ImplicationsforassessmentDay:Friday Time:3.40–4.10pm Room:OGGB5Following a rapid global progress anda reorientationof educational policies, each year,NorthAmericanuniversities hire an increasing number of foreign graduate students to teach undergraduate courses.TobetterunderstandthenatureofITAs’challenges,severalstudieshavebeenconductedoverthepastthreedecades.Theexistingresearch(Reinhardt2010;Chiang&Mi2008;Gorsuch2006;Hoekje&Williams1992;Briggs&Hofer1991),however,hasmainlybeenconductedintheEnglish-speakinguniversitiesintheUSandhasneglectedbilingualcontextssuchasthatofCanada,whereforeigngraduatestudentsoftenspeakneitheroftheofficiallanguagesastheirnativelanguage.
This study explores the language proficiency issues of ITAs enrolled in the engineering programs infrancophoneuniversitieswhere, inadditiontoproficiencyinFrench,anadvancedproficiencyinEnglishisintegral to ITAs’ academic and professional success. Adopting Bachman & Palmer’s model of LanguageKnowledge and Framework of Task Characteristics (2010), as well as Long’s (2005) model of needsassessment,thestudyusesamixed-methodsapproachtodatacollectionandanalysis.Data is gathered from 84 stakeholders (ITAs, their supervisors, and undergraduate students) usingquestionnaires,interviewsandobservationsinamajorfrancophoneuniversityinCanada.Theresultsofthestatisticalanalysisofdata(MANOVA)pointtotheinsufficientFrenchlanguageproficiencyoftheITAs.Theresults further confirm that, compared with the French language, the ITAs possess a higher level ofproficiencyinEnglish.Theresultsofthequalitativeanalysisofthedata,ontheotherhand,showamismatchbetween the language admission requirements of the graduate programs and the level of (French andEnglish)languageproficiencyrequiredoftheITAs.Basedonthesefindings,thecharacteristicsofthetasksandconstructstobemeasuredbypotentialadmissiontestsforITAstofrancophoneuniversitiesaredefined.
RuslanSuvorov(UniversityofHawai’iMānoa,UnitedStatesofAmerica)Test-takingstrategiesduringthecompletionofmultiple-choiceitemsfromtheMichiganEnglishTest:
EvidencefromeyetrackingandverbalreportsDay:Friday Time:11.20–11.50am Room:CaseRm2Thepastdecadeshavewitnessedasurgeofinterestinresearchontest-takingstrategiesinsecondlanguageassessment(e.g.Cohen1998;Kashkouli&Barati2013).UnderstandingstrategiesusedbyL2test-takerscanplay a critical role in validation research (Bachman 1990; Schmitt, Ng & Garras 2011) that has beentraditionallyrestrictedtotheuseofstatisticalmethods(O’Sullivan&Weir2011).Toinvestigatetest-takingstrategies,researchersusuallyemployconcurrentorretrospectiveverbalreports(e.g.Cohen&Upton2007;Plakans2009)thatarepronetoreactivityandveridicalityrisks(Bowles,2010)andshouldbesupplementedwith behavioural data that can provide information about test-takers’ actual engagement with L2 tasks(Brunfaut&McCray2015).
Thisstudyaimedatleveragingemergentmethodologythatcombineseyetrackingandretrospectiveverbalreportstoinvestigatestrategiesusedbytest-takersduringtheircompletionof58multiple-choiceitemsfromtheMichiganEnglishTest(MET).Usingtheconvergencemodelofthedatatriangulationdesign,itentailed:(a) Gatheringeye-movementdatafrom15non-nativespeakersofEnglishwhiletheywerecompletingthe
MET,and(b) Using eye-movement recordings as a stimulus for participants to describe test-taking strategies they
employedforansweringeachitem.
39
Descriptivestatisticsforglobalprocessingandtaskprocessingeye-trackingmeasureswereconvergedwiththemesidentifiedthroughretrospectiveverbaldataanalysistoprovideevidenceofthetypesoftest-takingstrategiesusedbyL2learners.Resultsrevealavarietyoftest-takingstrategiesforansweringmultiple-choiceitemsandindicatethattest-takers:(a) Differintermsofstrategiestheyemploytoanswersuchitems,and(b) Relyontest-wisenessstrategiesthattendtoinflatetestscores.Implications of the study suggest that themultiple-choice format appears to encourage the use of test-wiseness strategies thatmay introduce construct-irrelevant variance and pose threats to the validity ofproposedinterpretationsandusesoftestscores.
MikiTokunaga(FukuokaUniversity,Japan)EffectoftimepressureongrammaticalityjudgmenttestswithL1translation
Day:Friday Time:2.30-3pm Room:CaseRm2GrammaticalityJudgmentTests(GJTs)haveoftenbeenusedinSLAresearch(e.g.Ellis2009;Godfroid,etal.2015; Green& Hecht 1992; Roeher 2008; Sakai 2008; Shimada 2010).While timed GJTs are thought tomeasureconstructsrelatedtoimplicitknowledgeofthetargetlanguage,untimedGJTsareoftenpresumedtomeasureconstructsofexplicitknowledge.
Inthisstudy,timedanduntimedGJTswithL1(Japanese)translationsweregiventoJapaneseuniversityEFLlearners(n=219)toexaminewhethertimepressureinGJTswouldsignificantlyaffecttheperformance,thusindicating that timed and untimedGJTs possiblymeasure different factors of learners’ L2 knowledge orability.AlthoughGJTsinpreviousstudiesdidnotincludeL1translations,thisstudyattemptedtominimisetheeffectoflearners’vocabularyknowledgeandreadingability,andmeasuretheirunderstandingoftargetgrammarstructuresbyaddingJapanesetranslations.
RaschanalysisusingtheWinsteps®softwarepackagewasconductedonthedatafromthetests.Theresultsoft-testsandfactoranalysisindicatedthat,forthisgroupofparticipants,timepressuredidnotsignificantlyaffecttheresults.Theeffectofgrammaticalitywasmoresignificantthanthatoftimepressure, indicatingthatgrammaticalandungrammatical itemsonGJTsmaymeasuredifferentfactorsoflearners’L2.Amonggrammaticalitems,untimedgrammaticalitemsstoodouttobedifferentfromotheritemsincorrelationandfactoranalysis.Furtheranalysisisrequiredtofindoutwhethertheseitemsareinfactmeasuringsomethingdifferent,or judginggrammatical items,whichwasfoundtobeeasierthan judgingungrammatical items,withunlimited time, is simplymucheasier thanother item types,making themappear tobeadifferentcomponent.Theresultsofthestudy,alongwithadditionaldatatobecollectedthisyear,willbepresentedattheconference.
AlbertWeideman(UniversityoftheFreeState,SouthAfrica)TherefinementoftheideaofconsequentialvaliditywithinanalternativeframeworkforresponsibletestdesignDay:Saturday Time:1:05–1:35pm Room:CaseRm3Nationallevellanguageassessmentsposeaparticularchallengetothosewhodesignandadministerthem.Toillustratethatchallenge,thiscontributionwillconsiderasetofsecondaryschoolexit-levelexaminationsfor home languages in South Africa. These examinations illustrate a dilemma with such high-stakesassessmentsthatmaybeinformativeinothercasesaswell.Inordertoresolveit,therefinementoftheideaof consequential validity (Messick) will be considered from the vantage point of an alternativeconceptualisationoftheprinciplesthatinformthedesignoflanguagetests.
Thecontributionwilloutlinehowanumberofconstitutiveornecessaryconditionsforassessmentdesign(theirinstrumentalpower,theirconsistencyandtheirtheoreticaldefensibility)relatetoother,morerecentlyarticulatedideasandprinciplesoftestdesign.Themorerecentlyidentifiedissuesconcerntestaccessibility,acceptability, utility, alignment, transparency, impact, accountability, and care for those taking tests.This latter set of ideas may be defined as regulative or sufficient conditions for language assessments.Anemphasisonfairnessandjustice in languagetesting(Kunnan)makesanimportantcontributiontotheregulativeconditionsfortestdesign.
Theseconstitutiveconceptsandregulative ideas,andthedesignprinciplestheyreflect,willbe illustratedwithreferencetoworkonthepotentialredesignofasetofhomelanguageexitexaminationsinSouthAfricathat not only lack accountability and integrity in the public mind, but are also indefensible in terms of
40
adequacy and equivalence.While trade-offs and compromisesmay still have to bemade, a responsibleapproachtothedesignoflanguageassessmentwillmitigatethepotentiallynegativesocialandeconomicimpactofhigh-stakeslanguagetests.
JessicaWu(LanguageTestingandTrainingCentre,Taiwan)Evaluatingscorereportingpracticefortwolarge-scaleEFLtests:Intendedgoalandactualuse
Day:Friday Time:10:20–10:50am Room:CaseRm3Inresponsetothecallfortheincorporationofdiagnosticfeedbackintoachievementandproficiencytesting(e.g.Kunnan&Jang2009;Sawaki&Koizumi2015),somelarge-scaleEnglishlanguagetestshavestartedtoreport more detailed information about learners’ test performance. Yet, the success of the new scorereportingpracticedependsoneffectivecommunicationbetweenthetestdeveloperandthestakeholders.
Thissmall-scalequalitativestudydemonstrateshowonetestingbodyre-examinedtheeffectivenessofscorereportsfortwolarge-scaleEFLtestsinordertobetterbridgeassessmentandlearning.Thestudyinvestigatedthecurrentscorereportingpracticeandstakeholders’perceptionanduseofthedetailedfeedbackoftestresultsreportedfortheGeneralEnglishProficiencyTest(GEPT)andtheGeneralEnglishProficiencyEnglishTestforKids(GEPTKids).Threetypesofqualitativeanalyseswereconducted.First,acontentanalysisofthescorereportswasconductedbyemployingRobertsandGierl’s(2010)testscorereportanalysisframework.Second,thescorereportingpracticeofthetwotestswascomparedwiththatofotherlarge-scaleEFLtestswhich are also popular in Taiwan. Third, interviewswere conductedwith 20 students, 10 teachers, and6parents,whowerefromtheschoolswherethetwofocaltestswereused.Keyfindingsinclude:1. Thecurrentscorereportingpracticeofbothtestsisgenerallyconsistentwithgoodscorereporting
practiceidentifiedinpreviousstudiesineducationalassessment.2. Stakeholders’perceptionofthecontentandformatofthescorereportsforbothtestswasgenerally
favourable.3. Stakeholdersattendedtoandusedonlylimitedpartsofthereportedinformation,indicatingthatthere
isagapbetweentheintendedgoalandtheactualuseoftheinformationprovidedforsubsequentlearningandinstruction.
Implicationsforimprovinglanguagetestscorereportdesignandcommunicationoftestresultsarealsodiscussed.
MeganYucel(UniversityofQueensland,Australia)Narrativeinquiryinlanguageassessmentresearch
Day:Friday Time:12:30–1pm Room:CaseRm2The paper presents two narrative inquiry studies undertaken in different language assessment contexts,VietnamandAustralia,andwhichfeatureadiverserangeofparticipantsneedingtodemonstrateEnglishlanguageproficiencyforstudy,immigration,orprofessionalpurposes.Thestudiesinvestigatetheissueoftestimpactfromtheirperspective.Narrativeinquiryisaqualitativemethodologybasedonthepremisethatwemakesenseofourlivesthroughnarratives(Bruner1990).Thisapproachemphasisesthecollaborativeaspectofresearchbetweenresearcherandparticipantovertimeinasocioculturalcontext,andallowsforrichdescriptionandanexplorationofthemeaningsthatparticipantsderivefromtheirexperiences.Althougha substantial number of studies employing qualitativemethods in various areas of language assessmentresearchareavailable,narrativeinquirystudiesarerareinthefield.Consideringitsbenefitsofprovidingaholisticpicturewithrichinformationthatmaynotbecapturedinotherqualitativemethods,narrativeinquiryisparticularlyusefulforinvestigatingperspectivesofvariousassessmentstakeholders.
Thestudieswereconductedintwodifferentcontextsfocusingonteachersandlearnersrespectively.Study1investigatedthe impactofagovernment-mandatedEnglish languageproficiencystandardonVietnameseEFLteachersandexploredparticipants’perceptionsoftheirEnglishlanguageproficiencyincomparisonwiththe government standard. Study 2, inAustralia, investigated the beliefs of test candidates about Englishlanguagetestingwithanaimofprovidingfurtherinsightsintotestimpact,includingcandidates’perceptionsofIELTSanditsuseinmakingdecisionsaboutinternationaleducationandemployment.Thepaperreportsonthestudies’findings,whichprovidevaluableevidenceoftestvalidityandusefromtheperspectiveofthetest-taker,anddemonstratehowparticipants’narrativescanprovidecomprehensiveinformationessentialto interpret test results. The paper also discusses how the narrative approach could be used in futurelanguageassessmentresearch.
41
PAPERSINLANGUAGETESTINGANDASSESSMENT(PLTA)PapersinLanguageTestingandAssessment(PLTA)ispublishedbytheAssociationforLanguageTestingandAssessmentofAustraliaandNewZealand(ALTAANZ).Itoffersanopportunityforbothnewandexperiencedresearcherstopublishoriginalresearchpapers,essays/discussionpapersontheory,researchdigests,andbookandtestreviewsonlanguagetestingandassessmentissues.PLTAisapeerreviewedinternationaljournalandisoneofonlysixjournalsdedicatedexclusivelytolanguagetesting and assessment. PLTA is freely availableonline atALTAANZhttp://www.altaanz.org/ and at LTRChttp://ltrc.unimelb.edu.au/.Priorto2012,PLTAwaspublishedbytheLanguageTestingResearchCentreattheUniversityofMelbourneunder the nameMelbourne Papers in Language Testing. The full catalogue of back issues ofMelbournePapersinLanguageTestingcanbefoundatLTRChttp://ltrc.unimelb.edu.au/whereindividualpaperscanbedownloaded.PLTA is published annually or biannually. Enquiries can be directed to the Editorial Assistant, AnnemiekHuismanatplta.editor@gmail.com.CorrespondenceoneditorialmattersshouldbeaddressedtotheEditors,SallyO’HaganandLynMayatplta.editor@gmail.com.ThebookreviewseditorisAmandaMullerofFlindersUniversity.PLTABESTPAPERAWARDS2013–15Winner:Knoch,U.,&Elder,C.(2013).Aframeworkforvalidatingpost-entrylanguageassessments(PELAs).PapersinLanguageTestingandAssessment,2(2),48–66.Citation: This paper presents a significant and substantive step in thedevelopmentof PELAs (Post-entryEnglish Language Assessments) in Australia and New Zealand over the past 20 years. This paper is anextremely useful adaptation of the validity argument conceptualisation into a practical framework forvalidating PELAs. It provides broader applicability in terms of the discussion of validation/evaluationdistinction.TheframeworkwillnodoubtbeinfluentialintimeformanyinstitutionsdevelopingPELAs.Runner-up: Hudson, C., & Angelo, D. (2014). Concepts underpinning innovations to second languageproficiencyscalesinclusiveofAboriginalandTorresStraitIslanderlearners:adynamicprocessinprogress.PapersinLanguageTestingandAssessment,3(1),44–85.Citation:ThispapernotonlyhasaninvaluablecontributiontolanguageassessmentintheAustraliancontext,butalsopractical implicationsforothersimilarcontexts. Itdocumentsthedevelopmentofan instrumentwhichhasarisenoutofsocialandpedagogicalneedwithconsiderableinputfromclassroomteachers.Thispaper is anexcellent exampleofhowa rating scale can serveaprofessional development role andhowassessmentinstrumentsmightfitinthenexusofsecondlanguageacquisition,descriptivelinguistics,policyandeducation.OtherFinalists:Clark,M.(2014).Theuseofsemi-scriptedspeechinalisteningplacementtestforuniversitystudents.PapersinLanguageTestingandAssessment,3(2),1–26.Ruegg,R.(2014).Theeffectofassessmentofpeerfeedbackonthequantityandqualityoffeedbackgiven.PapersinLanguageTestingandAssessment,3(1),24–43.Kokhan,K.&Lin,C-K.(2014)TestofEnglishasaForeignLanguage(TOEFL):InterpretationofmultiplescorereportsforESLplacement.PapersinLanguageTestingandAssessment,3(1),1–23.
42
43
16PTE23_IntStudy_A4_AD_OL.indd 1 19/10/2016 6:32 PM
44
45
46
The British Council Assessment Research Awards and Grants Results for 2016
The British Council Assessment Research Awards and Grants recognise achievement and innovation within the field of language assessment and form part of the British Council’s extensive support of research activities across the world.
Assessment Research Awards and Grants Key dates for 2017 Call for proposals: November 2016 Closing date: 30 January 2017 Winners announced: March 2017
Assessment Research Awards These awards are designed to assist research students in their studies or in presenting their work at an international conference. The maximum award given is £2,500. Winners for 2016 are: Maria Georgina Fernandez Sesma (University of Southampton, UK, supervisor Dr Ying Zheng) Iftikhar Haider (University of Illinois at Urbana-Champaign, USA, supervisors Professor Emeritus Fred Davidson and Professor Melissa Bowles) Benjamin Kremmel (University of Nottingham, UK, supervisor Professor Norbert Schmitt) Suh Keong Kwon (University of Bristol, UK, supervisor Dr Guoxing Yu) Heidi Han-Ting Liu (Teachers College, Columbia University, USA supervisor Professor James E. Purpura) Yueting Xu (The University of Hong Kong, SARPRC, supervisor Professor David R. Carless)
Assessment Research Grants This grant scheme is designed to support projects that are directly focused on Aptis, the British Council’s English assessment tool. The maximum grant given is £17,500. Winners for 2016 are: Stephen Bax & Prithvi Shrestha (Open University, UK) for their project to explore lexical thresholds and lexical profiles across the Common European Framework of Reference for Language (CEFR) levels assessed in the Aptis test. Nguyen Thi Thuy Minh & Ardi Marwan (National Institute of Education, Nanyang Technological University, Singapore) for their project which seeks to analyse test-takers’ pragmatic performance and cognitive processing in the Aptis General Writing Test, Task 4. Sally O’Hagan & Kellie Frost (University of Melbourne) for their project which will examine test-taker processes and strategies and stakeholder perceptions of relevance of the Aptis for Teachers Speaking Test in the Australian context. Parvaneh Tavakoli & Fumiyo Nakatsuhara (University of Reading) for their project which looks at the scoring validity of the Aptis Speaking Test: Investigating fluency across tasks and levels of proficiency. Xun Yan, Ha Ram Kim & Ji Young Kim (University of Illinois at Urbana-Champaign) for their project which explores the complexity, accuracy and fluency features of speaking performances on Aptis across different CEFR levels.
www.britishcouncil.org/aptis/research
Innovation in Assessment Prize The Innovation in Assessment Prize celebrates innovation in the area of language testing and assessment. The winner for 2016 is the: Language Academy, Universiti Teknologi Malaysia, Johor Bahru, Malaysia, for their approach to developing an assessment instrument, the Test of English Communication Skills for graduating students (UTM-TECS), which measures university graduates’ readiness to communicate in English in the workplace. The tasks and assessment criteria of the test were derived from, and developed through, collaboration with industries at various stages of test development and implementation, including the validation and revision phases.
International Assessment Award This award recognises an individual working for the promotion of excellence in language assessment internationally. This year’s award is presented to Professor Emeritus Sauli Takala. Sauli Takala received his PhD from the University of Illinois at Urbana-Champaign in 1984. Sauli is now Professor Emeritus in Applied Linguistics at the University of Jyväskylä, Finland. He has extensive experience of research in language testing and assessment, in language policy and planning, curriculum development and teachers’ in-service education. For 15 years, he was on the Finnish Matriculation Examination Board. Sauli has published a large number of research reports and articles in Finnish, Scandinavian and international journals. He was editor of the Finnish Journal of Educational Research, co-editor of the Scandinavian Journal of Educational Research and is on the Editorial Board of Language Testing. Sauli coordinated the IEA International Study of Writing in the 1980s and helped plan the EU-funded internet-based DIALANG diagnostic assessment project in the 1990s. For many years, he has been associated with the Council of Europe’s work on modern languages, most recently with the Common European Framework of Reference for Languages (CEFR), in particular the Manual for Relating Language Examinations to the CEFR (2009). He is a founding member of the European Association for Language Testing and Assessment (EALTA), served on its Executive Committee and was its second President in 2007. He is a consultant for the European Centre for Modern Languages (ECML).
47
48
THEASSOCIATIONFORLANGUAGETESTINGANDASSESSMENTOFAUSTRALIAANDNEWZEALAND
Email:[email protected]:http://www.altaanz.org
Related Documents
