VALIDATION OF WORLD ENGLISH (WE) LISTENING COMPREHENSION SCALE by YASUSHI KYUTOKU Presented to the Faculty of the Graduate School of The University of Texas at Arlington in Partial Fulfillment of the Requirements for the Degree of MASTER OF SCIENCE IN PSYCHOLOGY THE UNIVERSITY OF TEXAS AT ARLINGTON MAY 2007
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
VALIDATION OF WORLD ENGLISH (WE)
LISTENING COMPREHENSION
SCALE
by
YASUSHI KYUTOKU
Presented to the Faculty of the Graduate School of
The University of Texas at Arlington in Partial Fulfillment
of the Requirements
for the Degree of
MASTER OF SCIENCE IN PSYCHOLOGY
THE UNIVERSITY OF TEXAS AT ARLINGTON
MAY 2007
ii
ACKNOWLEDGEMENTS
I want to make acknowledgments for my supervisor, Dr. Bernstein, for all the
supervision, instruction, advices, helps, kindness, patience, and encouragements in
order to complete my thesis research.
Also, I want to make acknowledgements for my committee members, Dr.
Kimball, Dr. Kopp, and Dr. Gorfein. Without your advices, suggestions, and helps, I
could not complete the thesis research. Again, thanks for devoting your precious time
for my thesis.
Also, I want to thank Dr. Dougall for advices, suggestions, and kindness.
Finally, I want to thank Charles Aden, Chawki Belhadi, Chikako Davidson,
Gabriel Davidson, Sachimi Kyutoku, Tomomi Ogura, and Ryo Yamada, and faculties,
staffs, and colleagues in our departments for inspirations and supports.
March 19, 2007
iii
ABSTRACT
VALIDATION OF WORLD ENGLISH (WE)
LISTENING COMPREHENSION
SCALE
Publication No. _____
Yasushi Kyutoku, M.S.
The University of Texas at Arlington, 2007
Supervising Professor: Bernstein, Ira H
World English (WE) includes any type of dialectical English spoken around the
world (Hinkel, 2005). In the present study, a WE listening test was developed. The
original form of the test consisted of six scales measuring, respectively, demographics,
exposure to non-native English speakers, self perception of WE fluency, WE listening
test, evaluations of non-native speakers' fluency, and attitudes towards non-native
English speakers. Internal consistency, dimensionality, correlations among scales, and
the effects of demographics on the test scores were analyzed based on the data collected
from 32 participants in Study 1. Results indicated moderate to moderately high internal
iv
consistency for the scales except for the attitudes toward non-native English speakers
scale. Study 2 constituted an actual computer administration with 106 participants. SE
listening comprehension test was added to the study. All of the scales except for the SE
test showed moderate to high consistency and unidimensionality.
v
TABLE OF CONTENTS
ACKNOWLEDGEMENTS...................................................................................... ii
ABSTRACT.............................................................................................................. iii
LIST OF ILLUSTRATIONS.................................................................................... ix
LIST OF TABLES.................................................................................................... x
the WE listening comprehension test, and Table 7.5 contains WE listening test for only
native speakers. Table 7.6 contains SE listening comprehension scale. Severe violation
of assumptions was not detected while screening data.
Table 7.1 Exposure scale
Variable N Mean SD rit
a1 104 3.3 1.8 .69
a2 104 3.0 1.8 .71
a3 104 3.9 1.1 .71
a4 104 3.6 1.3 .56
a5 104 1.7 1.3 .52
a6 104 1.6 1.3 .30
a7 104 3.2 1.3 .72
a8 104 3.0 1.2 .64
a10 104 3.0 1.3 .64
32
Table 7.2 Self-report scale Variable N Mean SD rit
b1 105 3.4 .86 0.67
b2 105 3.1 .87 0.79
b3 105 3.2 .93 0.72
b4 105 3.1 .94 0.65
b5 105 3.2 .90 0.69
b6 105 3.0 .91 0.79
b7 105 3.1 .93 0.83
b8 105 3.1 .92 0.69
b9 105 3.2 .93 0.70
Table 7.3 Evaluation scale Variable N Mean SD rit
j1 106 2.9 0.90 0.73
j2 106 2.7 0.82 0.74
j3 106 2.6 0.74 0.53
j4 106 3.0 0.79 0.47
j5 106 2.6 0.90 0.70
Table 7.4 WE listening comprehension scale Variable N Mean SD rit
t1 106 .86 .35 .40
t2 106 .88 .33 .36
t3 106 .44 .50 .25
t4 106 .51 .50 .29
t8 106 .92 .30 .24
t9 106 .74 .28 .30
t12 106 .72 .44 .23
t13 106 .72 .45 .23
t14 106 .34 .48 .29
t15 106 .91 .29 .25
t16 106 .75 .43 .21
t21 106 .68 .47 .21
t30 106 .87 .34 .23
t31 106 .52 .50 .27
t33 106 .41 .49 .29
t34 106 .55 .50 .30
t35 106 .74 .44 .29
t37 106 .75 .44 .30
t41 106 .42 .50 .29
33
Table 7.5 WE listening comprehension scale when only native speakers were included Variable N Mean SD rit
t1 90 .85 .35 .43
t2 90 .88 .32 .39
t3 90 .44 .50 .32
t4 90 .50 .30 .31
t8 90 .73 .28 .26
t9 90 .61 .44 .29
t12 90 .72 .45 .22
t13 90 .34 .45 .30
t14 90 .91 .48 .31
t15 90 .75 .29 .28
t16 90 .60 .43 .24
t21 90 .97 .47 .25
t30 90 .52 .34 .22
t31 90 .41 .50 .27
t33 90 .55 .49 .32
t34 90 .74 .50 .27
t35 90 .42 .44 .28
t37 90 .42 .44 .32
t41 90 .50 .50 .32
Table 7.6 SE listening comprehension test
Variable N Mean SD rit
item 1 106 .65 0.48 0.19
item 5 106 .58 0.50 0.19
7.2 The Effects of Demographics on WE Listening Comprehension Test
Demographic variables such as gender, native/non-nativity of English, and age
again did not related to scores on the WE listening test. As for the effect on WE test,
there were no trends of note the mean for males was 13.72 and the mean for females
was 13.52, and effect size was very small, Cohen's d = .059. It would require about 480
participants to achieve significant difference with α = .05, and power = .80. There were
34
no trends of note the mean for native English speakers was 13.64 and mean for non-
native speakers was 13.31, and effect size was very small, Cohen's d = .097. It would
require about 9000 participants to achieve significant difference with α = .05, and
power = .8. Since no significant effect of nativity on any scale based on one way
ANOVAs, data from both native and non-native participants were pooled (Table 7.7).
Table 7.7 Effects of demographics on listening scale
ANOVA
source df F p
Age 1 .57 0.45
Gender 1 .00 0.99
C.O.B. 1 .17 0.68
Error 100 (11.56)
Note: Values in parentheses are mean square errors
7.3 Validation of Scales
7.3.1 Exposure Scale
The result suggested moderately high internal consistency of the 9-item
Exposure scale (Cronbach’s α = .87, n =106. Item level analysis based on rit suggested
all items were related to the scale (Table 7.1) so all original items were retained. An
exploratory principal component factor analysis was then performed. The scale was
unidimensional based on parallel analysis criterion (Figure 7.1), and the resulting
pattern appears in Table 7.8.
35
Parallel analysis
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9
Number of factors
Eigen value
Scree plot
Randomized
Figure 7.1 PA for exposure scale
Table 7.8 Factor analysis for exposure scale
Factor Pattern Communalities
Items Factor 1 h2
1 .77 .60
2 .80 .64
3 .79 .69
4 .67 .56
5 .61 .76
6 .36 .66
7 .80 .66
8 .75 .56
10 .74 .54
36
7.3.2 Self-report Scale
The result suggested an acceptably high internal consistency for the 9-item self-
perceived competency scale (α = .92, n = 106). Item level analysis based on rit
suggested all items were related to the scale (Table 7.2), so all items were retained. An
exploratory principal component factor analysis was performed. The scale was
unidimensional based on parallel analysis criterion (Figure 7.2), and the resulting
pattern appears in Table 7.9.
Parallel analysis
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Number of factors
Eigen value
Scree plot
Randomized
Figure 7.2 PA for self-report scale
37
Table 7.9 Factor analysis for self-report scale
Factor Pattern Communalities
Items Factor 1 h2
1 .75 .56
2 .84 .71
3 .79 .62
4 .72 .52
5 .76 .57
6 .85 .72
7 .88 .77
8 .76 .57
9 .76 .58
7.3.3 Evaluation Scale
The result suggested moderately high internal consistency of the 5-item evaluation
scale (α = .83, n = 106). Item level analysis based on rit suggested all items were related
to the scale (Table 7.3), so all items were retained. The scale was unidimensional based
on parallel analysis criterion (Figure 7.3), and the resulting pattern appears in Table
7.10.
38
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5
Number of factors
Eiigen value
Scree Plot
Randomized
Figure 7.3 PA for Evaluation scale
Table 7.10 Factor analysis for evaluation scale
Factor Pattern Communalities
Items Factor 1 h2
1 .85 .72
2 .86 .74
3 .69 .48
4 .62 .38
5 .83 .69
7.3.4 WE Listening Comprehension Test
In order to achieve the final version of scale, there were four iteration processes.
This iteration process continued until the both the criterion for the Cronbach’s α =.7 and
rit = .2. First, items (6, 18, 23, 26, 28, 29, 32, 38, and 40) that had very low variance
were eliminated (eg. Everyone answered that question correctly). Then, items that had
lower than rit =.2 were eliminated in each step. In the second step, items 7, 10, 11, 17,
39
19, 20, 22, 24, 25, 26, 27, and 36 were eliminated. In the third step, item 43 was
eliminated. After the elimination, 20 out of 42 items were retained. The result
suggested moderate internal consistency (α = 71, n = 106). Item level analysis based on
rit suggested all retained items were related to the scale (Table 7.4). Internal consistency
of the WE listening comprehension scale for native speakers showed a similar result,
one that was not discussed here (Table 7.5). An exploratory principal component factor
analysis was performed. The scale was unidimensional based on parallel analysis
criterion (Figure 7.4), and the resulting pattern appears in Table 7.11.
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5 6 7 8 9
Number of factors
Eigne value
Scree plot
Randomized data
Figure 7.4 PA for WE listening comprehension test
40
Table 7.11 Factor analysis for WE listening comprehension test
Factor Pattern Communalities
Items Factor 1 h2
1 .54 .66
2 .52 .63
3 .37 .62
4 .40 .77
5 .36 .52
8 .43 .75
9 .36 .54
12 .53 .65
13 .32 .71
14 .38 .66
15 .34 .70
16 .30 .55
21 .30 .66
30 .31 .61
31 .37 .52
33 .39 .58
34 .42 .67
35 .44 .73
37 .43 .58
41 .39 .65
7.3.5 SE Listening Comprehension Test
There were originally 5 items on this scale, but only 2 items were retained due
to low internal consistency (α = .31, n =106, Table 7.6). Due to low internally
consistency and small number of items, meaningful analyses was not able to be
performed further. Low internal consistency seemed to be caused by both low
correlations among items and very small number of items in the scale. Based on the
Spearman-Brown prophecy formula, about 11 more items were needed to have a
reliability of .70.
41
7.4 Correlation among Scales
Descriptive statistics after the validation was shown in Table 7.12.
Table 7.12 Table of descriptive statistics
Scale mean SD Skewness Kurtosis
age 19.96 2.01 2.64 9.64
exp 26.18 8.86 0.09 -1.09
self 23.07 2.41 -0.69 0.026
eval 13.8 3.22 0.26 1.24
WE 13.6 3.38 -0.37 -0.33
SE 1.23 0.75 -0.42 -1.11
Results from correlation analyses (Table 7.13) suggested that there were a
medium positive correlation between the WE listening comprehension test and English
listening comprehension test, between the WE listening comprehension test and self-
report, and the listening test and the evaluation scale. The significant correlation
between WE test and self report was unexpected based on the study 1.
Table 7.13 Correlation among the scales Pearson r (N=106)
variable WE SE Self exp eval
WE __ 0.29** 0.36*** 0.02 0.22*
SE __ 0.05 0.03 -0.14
self __ -0.06 0.13
exp __ 0.09
eval __ ___
Note: WE = WE listening comprehension scale, SE = Standard English listening
comprehension test, Self = the Self report scale, Exp = the Exposure scale, and Eval =
the Evaluation scale. *P<.05, **P<.01, ***P<.001
42
CHAPTER 8
DISCUSSION (STUDY 2)
8.1 Validation of Scales
Except for SE test, results from the present study indicated moderate to
moderately high internal consistency for every scales. Based on the Spearman-Brown
prophecy formula, about 11 more items were needed to have a reliability of .70 for SE
test. Principal component analyses suggested the unideminensionality of these scales.
8.2 Correlation among Scales
Correlation analyses suggested that the self-report scale was significantly
correlated to performance. That implies that the self-report reflects the WE listening
scale. This conflicts with the findings from Study 1. However, self-report is still far
from accurate measure of WE listening skill because it accounted for only 13% of
variance of the WE test (R2 (106)= .13). Second, those who had higher scores on the
listening test evaluated the fluency of the non-native speaker in the test better than those
who scored lower on the listening test as was the case in Study 1. Significant positive
correlation between WE listening test performance and self-report scale seems to be
incompatible with those of the study 1. That might be attributed to the relatively
43
unreliable WE scale in Study 1. The fact that WE scale in study 2 appeared to be better
scale might be attributed to the modification of items and larger numbers of items in the
scale.
8.3 Effects of Demographics on WE Listening Comprehension Scale
Based on the ANOVA and power analyses, demographics such as gender, age,
and nativity of English did not have a significant effect on listening test scores. Indeed,
effect sizes of demographics were very small. Notably, native speakers did not
outperform non-native speakers in the WE listening comprehension scale as compared
with other SE listening comprehension scales such as the TOEFL (Stricker, 2004) as the
Study 1. However, because of the unequal number of participants in the native (90
participants) and non-native (16 participants) groups, this finding was not strongly
generalizable.
44
CHAPTER 9
GENERAL DISCUSSION
9.1 Conclusions, Limitations, and Future Directions
Because of internal consistency, unidimensionality, and a weak correlation
between the WE and the SE listening scale, WE listening comprehension scale appears
to be valid test to measure WE listening proficiency. However, some modifications
should be considered in further investigations. First, more items should be included in
the WE listening scale in order to make it more useful and reliable. Second, the SE test
should include more items to perform meaningful analyses. Third, a more basic level of
WE such as a word level or phoneme level should be analyzed in the investigation of
WE comprehension because it was difficult to analyze data in the conversation level.
Fourth, the low correlation between the exposure scale and WE listening test may be
caused by the construction of the exposure scale. In other words, the exposure scale
measured exposure to non-English speaking culture in general. Since this current study
included only a Japanese speaker in WE listening scale as a non-native English speaker,
exposure to Japanese English speakers rather than exposure to general non-native
speakers should be measured. Fifth, it will be interesting to use self-report scale after
the WE listening test as well as prior to WE listening comprehension test. Finally,
45
participants may have guessed what the non-native speaker said by using native
speaker's speech as contextual cue in the conversation sections. Thus, participants
might have used native speaker as contextual cue to answer the questions. Therefore,
listening skill to comprehend conversation between non-native speakers' English rather
than conversations between non-native and native speaker should be measured in
conversation section. Logically that will make the listening scale more internally
consistent because listening skill in the comprehension of only a non-native speaker is
measured in the monologue and the sentence sections as opposed to conversation part
that included a native speaker.
46
APPENDIX A
WE LISTENING COMPREHENSION SCALE IN STUDY 1
47
Please answer the following survey and the WE listening comprehension
scale as honestly and accurately as possible. Please answer all parts of the
following survey. In this survey, non-native English speaker is defined
anyone whose first or dominant languages is not English.
Demographic Survey
1. Age ( )
2. Gender ( )
3. Country of Birth ( )
Part A: Answer the following questions based on your experience.
1. How many non-native English speakers do you know?
( )
2. How long have you known them in years (list oldest ones)?
( ) years
3. How close you are to the one you consider yourself closest to?
Not close Very
At all Neutral Close
1 2 3 4 5
4. How often do you see that one person?
Very Very
Rarely Neutral Often
1 2 3 4 5
5. How many times have you visited countries in which English is not the dominant
language?
( )
6. How many days have you spent in countries in which English is not the dominant
language?
( )7. How familiar are you with non-American culture?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
48
8. How familiar are you with the conversation of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
9. How familiar are you with the writing of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
10. How fluent are you in a language other than English?
Not at Very
All Average Fluent
1 2 3 4 5
Part B: Imagine an average non-native English speaker, and answer
following questions.
1. Rate your attitude towards this person’s English fluency.
Not Very Very
Fluent Neutral Fluent
1 2 3 4 5
2. How favorable is your view of non-English cultures?
Very Neutral Very
Negative Positive
1 2 3 4 5
3. Do you want to go to a country in which English is not the native language?
Yes No
1 2 3 4 5
4. Do you want to live in a country in which English is not the native language?
Yes No
1 2 3 4 5
5. Are you interested in a non-American culture?
Yes No
1 2 3 4 5
49
6. Do you pay much attention to foreign cultures?
Yes No
1 2 3 4 5
7. When you do not understand a non-native speakers’ English, do you think that it is
usually something wrong with their fluency?
Yes No
1 2 3 4 5
8. When you do not understand a non-native speakers’ English, do you think that it is
usually something wrong with your listening skill?
Yes No
1 2 3 4 5
9. Do you regard Midwestern American or as the standard form of international
English?
Strongly Strongly
Disagree Neutral Agree
1 2 3 4 5
10. Do you regard British as the standard form of international English?
Strongly Strongly
Disagree Neutral Agree
1 2 3 4 5
Part C: Imagine an average non-native English speaker, and answer
following questions.
1. How good are you at understanding the English of someone who is not a native
English speaker?
Very Very
Poor Average Good
1 2 3 4 5
2. On average, how difficult is it for you to understand the overall English of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
50
3. How often do you find it difficult for you to understand the overall English of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
4. On average, how difficult is it for you to understand the English grammar of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
5. How often do you find it difficult for you to understand the English grammar of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
6. On average, how difficult is it for you to understand the English pronunciation of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
7. How often do you find it difficult for you to understand the English pronunciation
of someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
8. On average, how difficult is it for you to understand the English accent of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
9. How often do you find it difficult for you to understand the English accent of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
51
Before you start Part D, please listen to conversation 1. Then please
answer following questions as accurately as you can.
1. What was the customer looking for?
a) T-shirt b) Statue of Liberty Models c) Some souvenir d) Pewter
model
2. For whom did the customer want the product?
a) Mother b) Father c) Grandmother d) Herself
3. Why did the customer not want the first product the clerk recommended?
a) She did not like the design.
b) She did not like the material.
c) She did not have enough money.
d) She did not think it’ appropriate.
4. Which product was the customer first interested in?
a) T-shirt b) Statue of liberty model c) Silver ring d) Chocolate gift
5. Which word did the customer not understand?
a) Statue b) Copper c) Plastic d) Pewter
6. Which kind of material of the product did the customer buy?
a) Wood b) Copper c) Plastic d) Pewter
7. What kind of wrapping did the customer want?
a) Birthday b) Gift c) Christmas d) Souvenir
8. Which country did the customer come from?
a) Japan b) Jordan c) Korea d) China
9. How many times has the customer visited the U.S.?
a) 1 b) 2 c) 3 d) 4
10. How long does the customer plan to stay in the U.S.?
a) 3 days b) 5 days c) 7 days d) More than 7 days
11. When will the customer leave?
a) Today b) Tomorrow c) 2 days from now d) Next week
12. Other than New York, where did the customer visit?
a) Los Angels b) Philadelphia c) Pittsburgh d) Washington D.C.
52
13. How long did the customer spend there?
a) A day b) 3 days c) A week d) 10 days
14. What was the customer’s purpose of visit to the U.S.?
a) Business b) To see her mother c) To study d) For fun
15. Why did the customer not take the tour?
a) She could not afford it. b) She did not like it.
c) She did not know about it. d) She did not have a chance.
16. How did the customer like New York?
a) She liked it. b) She did not like it.
c) She did not mention like or dislike. d) She liked D.C. better.
17. Which show did the customer watch?
a) Phantom of the Opera b) Miss Saigon c) Cats
d) She did not watch any.
18. Why did the customer go to Hard Rock Café?
a) It is her favorite place.
b) It was her first time to visit there.
c) She likes burger there.
d) She likes their goods.
Before you start Part E, please listen to Monologue 1. Then please answer
following questions as accurately as you can.
1. What is her occupation?
a) Instructor b) Counselor c) Professor d) Principal
2. Where did her husband come from?
a) Africa b) Japan c) Canada d) The U.S.
3. Why was she reading a book about cats?
a) She likes cats.
b) She is studying about cats.
c) It is her assignment.
53
d) She has a cat.
4. What is her dream?
a) To live in foreign countries b) To be a teacher
c) To marry her boy friend d) To write a book
Before you start Part F, please listen to Monologue 2. Then, please answer
following questions as accurately as you can.
1. According to the conversation speaker’s friend Yoko lives in.
a) America b) Russia c) Asia d) Europe
2. How does the speaker evaluate her friend Yoko?
a) She thinks Yoko is a great person.
b) She does not like Yoko.
c) Yoko is nice, but she has to develop more social skills.
d) Yoko cannot speak French.
3. What is the most difficult part of living in Switzerland?
a) She does not like winter.
b) She does not like the area where she lives.
c) Living costs are too expensive where she lives.
d) She does not like to work hard.
4.Which country did the speaker imply Yoko comes from?
a) Japan b) The U.S.A. c) China d) Switzerland
54
Before you start Part G, please listen to Sentences.
1. Why did she not go to the concert?
a) Tickets are too expensive
b) She did not like the band.
c) She did not have the time.
d) She could not get a ticket.
2. Why did she not go to John’s house?
a) It is too far for her. b) It is too cold.
c) She does not like John d) She does not have time to go.
3. How is she regarding Pennsylvania?
a) She is not there right now. b) She is there.c) She will live there. d) She
wishes to live there.
4. What does the sentence imply about Mary
a) One should feel sorry for her.
b) One should apologize to her.
c) One doesn’t need to apologize to her.
d) She should be ignored. .
Part H: Please answer the following questions.
1. Rate the speaker’s fluency in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
55
2. Rate the speaker’s pronunciation in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
3. Rate speaker’s accent in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
4. Rate the speaker’s grammar in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
5. Rate the speaker’s clarity in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
56
APPENDIX B
WE LISTENING COMPREHENSION SCALE IN STUDY 2
57
Appendix B: Modified WE Scales
Please answer the following survey and the WE listening comprehension
scale as honestly and accurately as possible. Please answer all parts of the
following survey. In this survey, non-native English speaker is defined
anyone whose first or dominant languages is not English.
Demographic Survey
1. Age ( )
2. Gender ( )
3. Country of Birth ( )
Part A: Answer the following questions based on your experience.
1. How many non-native English speakers do you know?
1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11
2. How long have you known them in years (list oldest ones)?
1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11
years
3. How close you are to the one you consider yourself closest to?
Not close Very
At all Neutral Close
1 2 3 4 5
4. How often do you see that one person?
Very Very
Rarely Neutral Often
1 2 3 4 5
5. How many times have you visited countries in which English is not the dominant
language?
1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11
58
6. How many years have you spent in countries in which English is not the dominant
language?
1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11
years
7. How familiar are you with non-American culture?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
8. How familiar are you with the conversation of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
9. How fluent are you in a language other than Standard English?
Not at Very
All Average Fluent
1 2 3 4 5
Part B: Imagine an average non-native English speaker, and answer
following questions.
1. How good are you at understanding the English of someone who is not a native
English speaker?
Very Very
Poor Average Good
1 2 3 4 5
2. On average, how difficult is it for you to understand the overall English of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
3. How often do you find it difficult for you to understand the overall English of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
59
4. On average, how difficult is it for you to understand the English grammar of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
5. How often do you find it difficult for you to understand the English grammar of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
6. On average, how difficult is it for you to understand the English pronunciation of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
7. How often do you find it difficult for you to understand the English pronunciation
of someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
8. On average, how difficult is it for you to understand the English accent of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
9. How often do you find it difficult for you to understand the English accent of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
Before you start Part C (question 1-17), please listen to Part C. You
may listen to the Part C only once. Do not look at or start answering
the questions until you finish listening Part C.
60
1. What was the customer looking for?
a) T-shirt b) Statue of Liberty Model c) Some souvenir d) Specific
souvenir
2. For whom did the customer want the product?
a) Mother b) Her friend c) Grandmother d) Herself
3. Why did the customer not want the first product the clerk recommended?
a) She did not like the design.
b) She did not like the material.
c) It was too expensive.
d) She did not think it’s appropriate.
4. Which product was the customer first interested in?
a) T-shirt b) The medium model c) Silver ring d) The large
model
5. Which word did the customer not understand?
a) Statue b) Copper c) Souvenir d) Pewter
6. Which kind of material of the product did the customer prefer?
a) Pewter b) Copper c) Plastic d) None of the above
7. What kind of wrapping did the customer want?
a) Birthday b) Gift c) Christmas d) Paper
8. How many times has the customer visited the U.S.?
a) 1 b) 2 c) 3 d) 4
9. How long does the customer plan to stay in the U.S.?
a) 3 days b) 5 days c) 7 days d) More than 7 days
10. When will the customer leave?
a) Today b) Tomorrow c) 2 days from now d) Next week
11. Other than New York, where did the customer visit?
a) No where b) Hard Rock Cafe c) Broadway d) Washington D.C.
12. How long did the customer spend there?
a) A day b) 3 days c) A week d) 10 days
13. What was the purpose of visiting there?
a) For shopping b) For museum c) For sightseeing d) For the
tour
61
14. What did she buy there?
a) Souvenir b) Ticket c) Food d) A badge
15. What was the customer’s purpose of visit to the U.S.?
a) Business b) To see her mother c) To shop d) For fun
16. Why did the customer not take the tour?
a) She could not afford it. b) She wanted to go by herself.
c) She did not know about it. d) She did not have a chance.
17. Where she did not go?
a) Steak house b) Museum
c) White House d) Statue of Liberty
18. How did the customer have fun in New York?
a) Food b) Musical c) Sightseeing d) Museum
19. Why did the customer go to Hard Rock Café?
a) It is her favorite place.
b) It was her first time to visit there.
c) She likes burger there.
d) She likes their goods.
Before you start Part D (question 20-27), please listen to Part D. You
may listen to the Part D only once. Do not look at or start answering
the questions until you finish listening Part D.
20. What did she expect that his view of Japanese food?
a) Delicious b) Too heavy c) Too light d) Too salty
21. What did she think about Japanese foods in the U.S.?
a) Delicious b) Too heavy c) Too light d) Too salty
22. What did they disagree about the tastes of rice and tofu?
a) He insisted rice and tofu have taste. b) She insisted that rice and tofu
have taste.
c) She does not like rice and tofu. d) She does not like rice and tofu.
62
23. How did she think about his sense of tasting foods?
a) He does not have good sense of tasting any foods.
b) He has good sense of tasting any foods.
c) He does not have good sense of tasting Japanese food.
d) He has a good sense of American foods.
24. How did she evaluate popular American-Japanese food?
a) Delicious b) Too heavy c) Too light d) Too salty
25. Which part of his explanation of difference between Japanese and American recipes
she did not understand?
a) Metric system b) Ingredients c) Type of a food d) Cooking
procedure
26. What was her example of wrong American food in Japan?
a) American coffee b) American tea c) American burger d) American
juice
27. How did they settle down about argument?
a) They accepted differences between cultures. b) They did not ague.
c) They changed topic. d) He accepted her view.
Before you start Part E (question 28-31), please listen to Part E. You
may listen to the Part E only once. Do not look at or start answering
the questions until you finish listening Part E.
28. What is her occupation?
a) Instructor b) Counselor c) Professor d) Principal
29. Where did she get married?
a) Africa b) Japan c) The U.S. d) None of the above
30. Why was she reading a book about cats?
a) She likes cats.
b) To write a book.
63
c) It is her assignment.
d) She has a cat.
31. What is her dream?
a) To write a book about cats b) To be a teacher
c) To marry her boy friend d) None of the above
Before you start Part F (question 32-33), please listen to Part F. You
may listen to the Part F only once. Do not look at or start answering
the questions until you finish listening Part F.
32. Why did her husband wanted to see the chef?
a) Because he is a great chef. b) Because he is a TV star.
c) Because he is a famous chef. d) The speaker did not mention why.
33. What kind of restaurant was it?
a) Japanese b) American c) Chinese d) Korean
Before you start Part G (question 34-37), please listen to Part G. You
may listen to the Part F only once. Do not look at or start answering
the questions until you finish listening Part F.
34. According to the conversation speaker’s friend Yoko lives in.
a) America b) Russia c) Asia d) Europe
35. How does the speaker evaluate her friend Yoko?
a) She thinks Yoko is a great person.
b) She does not like Yoko.
c) Yoko is nice, but she has to develop more social skills.
d) Yoko cannot speak French.
64
36. What is the most difficult part of living in Switzerland?
a) She does not like winter.
b) She does not like the area where she lives.
c) Living costs are too expensive where she lives.
d) She does not like to work hard.
37. Which country did the speaker imply Yoko comes from?
a) Japan b) The U.S.A. c) China d) Switzerland
Before you start Part H (question 38-42), please listen to Part H. You
may listen to the Part H only once. Do not look at or start answering
the questions until you finish listening Part H.
38. Why did she not go to the concert?
a) Tickets are too expensive.
b) She did not like the band.
c) She did not have the time.
d) She could not get a ticket.
39. Why did she not go to John’s house?
a) It is too far for her. b) She can go there any time.
c) She does not like John d) She does not have time to go.
40. How is she regarding Pennsylvania?
a) She is not there right now. b) She is there.
c) She will live there. d) She wishes to live there.
65
41. What does the sentence imply about Mary?
a) One should feel sorry for her.
b) One should apologize to her.
c) One doesn’t need to apologize to her.
d) She should be ignored.
42. According to the speaker
a) This fall will be colder.
b) Usually, it is colder in fall.
c) It is a nice day.
d) She complaints about warm weather.
Before you start Part I (question 43-), please listen to Part I. You may
listen to the Part H only once. Do not look at or start answering the
questions until you finish listening Part H.
43. Why does the second speaker wanted to go to England?
a) Because it was on magazine.
b) To see gardening plant.
c) To see a person.
d) Because of special event.
44. Why does the first speaker want to go to Cambodia?
a) Because it was on magazine.
b) To see gardening plant.
c) To see a person.
d) Because of special event.
45. Which country was not mentioned by them?
a) Japan
b) England
66
c) Jordan
d) Iran
46. Which places they agreed that they want to go?
a) Parts of the U.S.
b) Asian countries
c) Egypt
d) Both a and c
47. Which place at least one of them has been to?
a) Cambodia
b) Egypt
c) Hawaii
d) England
Part J: Please answer the following questions.
1. Rate the non-native speaker’s fluency in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
2. Rate the non-native speaker’s pronunciation in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
3. Rate the non-native speaker’s accent in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
67
4. Rate the non-native speaker’s grammar in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
5. Rate the non-native speaker’s clarity in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
68
APPENDIX C
WE CONVERSATION SCRIPTS
69
Conversation 1
A: Hi, may I help you?
B: Oh, yes please.
A: Are you looking for something in particular?
B: Yes. I want to buy a souvenir for my mom.
A: How about an “I love New York T-shirt” ?
B: Well, my mom doesn’t wear T-shirts. Do you have anything else?
A: I see. We have some Statue of Liberty models.
B: It sounds good. Can I see them?
A: Sure, just over here. We have a lot of different sizes.
B: .........
A: Do you see any that you like?
B: Mmm, this middle size one looks good for her. I’ll take this one.
A: O.K. That one’s pewter, so it’s 15 dollars.
B: What’s pewter?
A: Like silver but soft.
B: Do you have any different materials?
A: Of course. We have gold, silver, wood, copper, plastic. What would you like?
B: Which do you recommend?
A: This copper one is the most realistic.
B: How much is it?
A: This one’s 12 dollars.
C: O.K. I’ll take it.
G: Would you like that gift-wrapped?
C: Oh, can you? Please.
A: No problem. Birthday paper or Christmas paper or anniversary or just normal?
B: Well, Christmas one, please.
A: Sure. Just a moment, please.....So, where are you from?
70
B: I'm from Japan. Have you been to Japan?
A: No, I haven't. Is this your first time to America?
B: Yes. I've been here for 10 days, and I'll leave the day after tomorrow.
A: Are you here on vacation?
B: Yes, I am.
A: Have you spent the whole time in New York, or did you go anywhere alse?
B: I went to Washington D.C.
A: How long did you spend there?
B: For 3 days. I went to see the White House.
A: DId you take a tour?
B: I wanted to. But I couldn' t make a reservation.
A: Oh well, did you go to the Smithsonian?
B: Well I don't know it.
A: It's the big group of museums near the capital building.
B: O.K. I'll go there when I go to Washington D.C. next time.
A: So, you spent the other 7 days in New York?
B: Yes. I enjpyed New York very much.
A: Thanks. So, did you see a Broadway show?
B: Yes, I saw "Cats". It was wonderful.
A: Yeah,everyone likes that one. Are you stying in Manhattown?
B: Yes.
A: Did you go to the Hard Rock Cafe?
B: Yes. I like Hard Rock Cafe. I collect their badges.
A: How many do you have?
B: I have 7. 5 are fom Japan,1 is from Washinton D.C. 1 is from New York.
A: Are you from Tokyo?
B: No, I'm from Osaka, third largest city in Japan.
A: Oh, I'm sorry. O.K. Here you go. One gift-wrapped Statue of Liberty.
71
B: O.K. Here you go.
A: Thank you, and 3 dollars is your change. Have a nice day.
b: Thank you.
72
Conversation 2
A: What did you think the first time you ate Japanese food?
B: Ahh, let’s see, that was when I was in high school… I thought “Wow, this is good. It’s kind of like Chinese food.”
A: Didn’t you think that Japanese food’s flavor was light?
B: Not really, it was pretty salty and strong.
A: Because you had it in America. When I went to America, I thought “This isn’t real Japanese food.”
B: Yeah but I’ve had Japanese food in Japan that’s had pretty strong flavors, haven’t you?
A: Yeah, some are salty. But when I had food in America, everything tasted heavy. Actually your cooking tastes a little heavy.
B: That’s just because you’re used to bland foods like rice and tofu.
A: You usually say that rice and tofu have no flavor but they do! I can tell! I can taste it!
B: It’s just your imagination.
A: You can ask your students! I’m sure they’ll agree with me. And also, water has flavor.
B: Only if the flavor is added, like that C1000 Lemon Water.
A: You don’t understand.
B: Don’t understand what?
A: Even though you like cooking and going to good restaurants, why can’t you tell their flavor? You can tell Coke’s flavor at McDonald’s and so on. Like the syrup is too little or too much.
B: That’s different, that’s like how you can tell the difference between my ribs with five spice powder and the ribs without it.
A: OK, you’ll understand some day, I hope. By the way, what did you have the first time you had Japanese food?
B: I had teppanyaki, at a restaurant in Fort Wayne (the city we lived in in Indiana)..
A: Teppanyaki isn’t so Japanese for me. I know it’s popular in America but Japanese food is tempura or sushi for most Japanese. Isn’t tempura famous in America?
B: It’s kind of famous but less than sushi or teppanyaki. I had heard of sukiyaki and teriyaki in America, too. I had teriyaki before coming to Japan but not sukiyaki.
Anyway don’t you think saying Japanese food is sushi and tempura is stereotyping? Don’t you want other Japanese food to be famous?
73
A: Yes, other Japanese food can be famous, but also I’m proud of sushi and tempura. What Japanese food can be popular in America you think?
B: I think fried noodles and okonomiyaki could be popular. I think yakiniku could be really popular, too, but it probably won’t have a chance because of too many safety regulations.
A: Yeah, okonomiyaki is very good, and battered octopus balls. OK, we can open a restaurant in America!
B: With okonomiyaki and octopus balls? No way, too Osaka! How about Nagoya-style wings and noodles? You know the wings would be popular, remember when I brought them to that party?
A: I liked your wings, but you know I don’t like Nagoya so much. By the way, you read recipes in Japanese and in English. Is there any difference between them?
B: Only in the names of the ingredients and in the measurements, since Japan uses the metric system.
A: What’s the metric system?
B: You know, meters, grams, liters…
A: What system does America use?
B: We use the English system. Feet, pounds, ounces… Anyway, you know a lot of so-called American food in Japan is wrong too, right?
A: You mean American coffee? Light-flavored?
B: That’s one example. It’s like hot brown water, real American coffee is definitely not like that.
A: Things from overseas can be different in different countries.
B: So you shouldn’t complain about Japanese food in America then, right?
A: OK, I’ll try to not complain.
74
Monologue 1
Hello. I am going to tell about myself. I am a teacher for a cram school. Teaching is
interesting for me. And I am married. My husband is American. His name is Gabriel.
He is very sweet. And I like reading books. Now I am reading a novel about cats. I like
cats. Sometime I want to have cats. At last, I will tell you about my dream. I want to be
a translator someday. I want to publish my books. Thank you.
Monologue 2
Hello, I am going to tell you about my experience. Last Tuesday was my husband’s
birthday. So, we went to a good restaurant in Nagoya last Sunday. It’s a famous chef’s
Chinese restaurant. His name is Chin Kenichi. He became famous from Japanese TV
program, Iron Chef. We really enjoyed lunch. When we paid, my husband asked the
staff if Mr. Chin was there. But, later the staff came to us and said “it’s actually secret,
but if you wan to see Chin, He will be here on December 19th and 20
th. We were very
happy to hear it. Probably my husband will go there that day. Thank you.
Monologue 3
Hi, I will tell you about my friend Yoko. I’ve known her for 5 years. Now, she lives in
Switzerland because her husband is Swiss. It can be difficult to live in foreign
countries. And, so it is for Yoko. First, she speaks French there. She had to study
French very hard. Second, it’s very cold in winter and winter starts much faster then in
Japan. Third, everything is expensive there. And she doesn’t like the new or different
things. So it is difficult to get Japanese things. There are some more reasons, but Yoko
always does her best. She never gives up. So, I really respect her. Thank you.
Sentence 1
I would have gone to the concert if I had had a ticket.
Sentence 2
I don’t like to go to John’s house because it’s 5 miles from here.