Vocabulary Testing: Diagnose, Track Growth, & Intervene Kathleen T. Williams, PhD [email protected].

Vocabulary Testing: Diagnose, Track Growth, & Intervene

Kathleen T. Williams, [email protected]

Snapshot of Speaker

• Began professional life as a second grade teacher in 1968

• Involved in test and instructional material development since 1989

• Has been a speech pathologist, classroom teacher, school psychologist, graduate & undergraduate instructor

• Has authored tests for vocabulary, reading, and mathematics

• Has a BS in speech pathology, MEd in elementary education, MA in educational psychology, and PhD in school psychology

Overview• Importance of assessing vocabulary• Conorming tests• Understanding scores

– Normative or derived scores– Statistical significance of score difference– Prevalence of score difference – Tracking growth

• Generating hypotheses– Receptive > Expressive– Expressive > Receptive

• Qualitative analysis for designing intervention strategies

What is “Vocabulary”?• Lexical Store

– Mental dictionary of known meanings of words and idioms

– Some may be inaccurate or incomplete

• Recognition vocabulary– For listening and reading

• Productive vocabulary– For speaking and writing

• Not the number of words one can decode in print• It is the concepts one can understand at some

level

Levels of Word Knowledge

• Unknown– Totally unfamiliar with the word– Never hear it– Never saw it in print

• Personal example:– carapace– “under the cold carapace of her adult

disappointments”– a protective, decorative, or disguising shell

Levels of Word Knowledge

• Acquainted– Some familiarity with the word– May have seen or heard the word– Do not clearly understand it’s meaning– May understand its meaning in only one context– Could not use it precisely in writing or speaking

• Personal example:– hessian– “The scullery maid backed away, her head down,

wiping her dirty hands on her hessian apron.”– burlap

Levels of Word Knowledge

• Unknown– Totally unfamiliar with the word– Never hear it– Never saw it in print

• Acquainted– Some familiarity with the word– May have seen or heard the word– Do not clearly understand it’s meaning– May understand its meaning in only one context– Could not use it precisely in writing or speaking

• Well known– Can comprehend the word when reading– Can comfortably use the word when speaking or writing– Understand its multiple means and uses

The Importance of Vocabulary

• Testing vocabulary is part of most preschool screening batteries and Reading First K to 3 programs

• Without a basic vocabulary, the potential for developing a reading problem is great

• Not all children come to school with the basic vocabulary knowledge required to understand the directions and instructions of the classroom

Limited Vocabulary – Lasting Consequences

• Hart & Risley (1995) demonstrated that early differences noted in children entering school remain static throughout their education

• Many children with an underlying language disability also demonstrate poorly developed oral vocabularies

• Plays a vital role in the development of an individual’s academic and career opportunities

• Vocabulary knowledge is strongly related to overall reading comprehension

–Correlations range from .85 to .90

• If a word is decoded and pronounced but the meaning is not recognized, comprehension will be impaired

• A limited vocabulary represents a limited understanding of concepts

• Well-developed vocabulary skills and wide background knowledge help individuals comprehend more difficult and complex material

• The more words you know, the more you can read

• The more you read, the more words you will learn

• For children starting school with a limited vocabulary, “more” reading does not result in “more” vocabulary (Stanovich, 1986)

PPVT-4 & EVT-2• Conormed measures of receptive & expressive

vocabulary• Ages 2 ½ through 90+• PPVT-4 (Peabody Picture Vocabulary Test, Fourth

Edition, Dunn & Dunn, 2007)– Plate of four pictures– Examiner says a stimulus word– Examinee points to one picture

• EVT-2 (Expressive Vocabulary Test, Second Edition, Williams, 2007)– Single picture– Examiner asks a question– Examinee says one word that is a label or a synonym

• Two parallel forms to each test

Conorming

• Standardized on the same normative sample– Same population– Same point in time– Robust and stable measures of the

relationship

• Correlation of EVT-2 to PPVT-4 is .82

• Shared variance is 67%

Understanding Scores

• Limitations of raw scores

• Normative or derived scores

• Interpreting standard scores, percentiles, normal curve equivalents, and stanines

• Limitations of Age or Grade Equivalents

• Monitor progress or track growth with GSVs

• What to do with extreme scores

Limitations of Raw Scores

– Not comparable from one test to another– Tests have different numbers and types of

items• PPVT-4: 228 receptive items• EVT-2: 190 expressive items

– May have similar but unique growth curves and varying score distribution

– Within a test, the same raw score does not tell the same story

Normative or Derived Scores• Reference a examinee’s performance to a

single group of interest

• For PPVT-4 & EVT-2, you have a choice:– The referenced group can be based on the

individual’s age (2 ½ through 90), or– The reference group can be based on grade

and time of school year (fall or spring)

• Normative scores include standard scores, percentile ranks, normal curve equivalents, and stanines

When to Use Age-based Scores

• Test results will be used as part of an evaluation that considers age-based norms as best practice, such as pre-referral evaluation as required by IDEA

• An examinee is not of the typical age range for his or her same-grade peers due to retention or other factors

• Testing is being completed in an atypical educational setting, such as a summer school program

• The formal educational background of the examinee has been sporadic or uneven for some reason, such as illness or transience

Interpreting Standard Scores

• Indicates the distance of a examinee’s raw score from average, taking into account the variability of scores among examinees of that same age or grade

• Mean of 100 and a standard deviation of 15– SS of 85 to 115 = 68 percent of the group– SS below 85 or above 115 = ~16 percent– SS below 70 or above 130 = ~2 to 3 percent

• Equal interval scale of measurement– Can add, subtract, multiple, & divide

• Must get the same standard score next year to demonstrate a year of growth

Interpreting Percentiles

• Percentile rank indicates the percentage of examinees in the reference group who performed at or below a specific examinee’s score– Percentile of 86 means the examinee scored as

high or higher than 86 percent of the normative sample

• Percentiles are an ordinal or rank-order scale of measurement– Not an equal-interval scale of measurement– Can not add, subtract, multiply, or divide

Normal Curve Equivalents

• Referred to as NCEs

• Range from 1 to 99, with a mean of 50 and a standard deviation of 21.06

• Are based on percentiles but have been statistically converted to an equal-interval scale of measurement– Can be averaged– Useful for reporting data

Interpreting Stanines

• Stanines is a contraction of “standard nines.”

• Single digit scoring metric with a mean of 5 and a standard deviation of 2

• Not an equal-interval scale of measurement

• They provide a “band” interpretation rather than a single-score cut-off.

• Look for stanines that represent different sections of the normal distribution

Developmental Norms

• Grade equivalents (GE) or age equivalents (AE) place an examinee’s performance along a growth curve across grade or age

• A GE or an AE is the grade or age at which a particular raw score is the average score

• GEs are expressed in tenths of a grade

• AEs are expressed by years and months

Limitations of Grade or Age Equivalents

• An GE or AE does not necessarily mean that the examinee’s vocabulary knowledge is qualitatively the same as that of the average person at that grade or age

• Should not be interpreted as a functional equivalent (different raw scores can indicate different strengths and weaknesses)– An examinee aged 12 with an AE of 9:6 may tend to know a

different set of words than the average 9-year-old

• Are not an equal-interval scale of measurement– Place examinees along a development continuum,

which may not increase at regular intervals• Should not be used for diagnostic or placement

decisions

Understanding Growth Scale Values • Growth scale values (GSVs) are included for both

PPVT-4 & EVT-2• Developed so progress could be followed over a period

of years on the same continuous scale– Standard scores, percentiles, NCEs, and stanines place an

examinee’s vocabulary knowledge with a reference group (based on all those of the same age or grade)

• The GSVs provide a measure of an examinee’s vocabulary knowledge in reference to the entire range of growth across all ages or grades – As an individual’s vocabulary grows, the GSV will increase

• An equal-interval scale– Can be added, subtracted, or averaged– Can be compared over time for many purposes such as RTI or

program evaluation– Allows tracking of individual or group progress

Tracking Growth - Example

Age RS SS GSV

4:4 33 80 112

5:5 49 82 126

6:4 60 82 135

7:4 73 86 145

8:4 86 84 154

What to Do with Extreme Scores

• Very low scores– No interpretation of a raw score of 0 on EVT-2

• It is lower than a score of 1, but how much can not be determined

– No interpretation of a raw score of 3 or lower on PPVT-4

• Corresponds to a score an examinee would obtain through random responding

• Perfect or near perfect scores– Should be interpreted with caution

• Report SS of 160 but note limitation on record form– Examinee’s vocabulary may not have been measured fully

by the test

Qualitative Analysis of Results

• Calculate the absolute value of the difference between PPVT-4 and EVT-2 standard scores

• Determine a level of significance of the difference (Table B.7 in EVT-2 Manual)– NS, .15, .10, .05, or .01– Example: If significant at .05, 95% probability did not

occur by chance

• Find the percentage of the normative sample with the obtained difference value (Table B.8 in EVT-2 Manual)

Example 1

• Age 7 years 1 month• PPVT-4 SS of 78• EVT-2 SS of 84

– 6-point difference, not significant• Could have occurred by chance

– Prevalence of difference• Occurred in more than 25% of the sample

• Both SSs reflect a below-average performance

Example 2

• Age 8 years 8 month• PPVT-4 SS of 82• EVT-2 SS of 73

– 9-point difference, significant at the .10 level• 90% probability did not happen by chance

– Prevalence of difference• Occurred in over 25% of the sample• Fairly common

• Both SSs reflect a below-average performance

Example 3

• Age 6 years 6 month• PPVT-4 SS of 81• EVT-2 SS of 101

– 20-point difference, significant at the .01 level• 99% probability did not happen by chance

– Prevalence of difference• Occurred in only 5% of the sample• Fairly uncommon

• Receptive score below average but expressive score average

Receptive > Expressive

• Good guesser; may have some knowledge of the word and can rule out one or two of the options

• Good lexical store, but a problem with word retrieval

• Broad experiential background; knows many labels

• Strong knowledge of the prevailing culture; knows the current label

Expressive > Receptive

• Lack of automaticity of word recognition skills

• Deep processing vs. superficial

• Limited experiential background; knows only one label

• Different cultural or “world knowledge” base; can say an acceptable word

• History of hearing impairment; has learned a specific set of labels

•“The greatest mistake we can make, I believe, is to treat comprehension as a simple process.”

–Theory, Assessment, and Intervention in Language Disorders: An Integrative Approach, by Elizabeth Carrow-Woolfolk, 1988

Qualitative Analysis

• Home versus school vocabulary

• Vocabulary by part of speech

• Crossover vocabulary sampling between the EVT-2 and PPTV-4

• Three-tier model

Home Versus School

• Home words– Words of high or moderately high frequency

that could be acquired through common life experiences

– Labels children hear in a home environment rich in language experiences

• Frequently read to• “Lots of talk”

Home Versus School

• School words– Not common to home environments– Included in instructional lessons about exotic

or foreign places– Labels for less concrete objects (geometric

shapes)– Learned by reading newspapers, magazines,

or other informational materials

Home Versus School

• Home– Household objects & food items– Common objects found outside the home– Basic numbers & colors– Common domesticated & wild animals– Actions (verbs)– Attributes (adjective & adverbs)– Body parts

Home Versus School

• School– Less common domesticated & wild animals– Geometric shapes– Musical instruments– Names of workers or occupations– Recreational items not found in all homes– Items specific to urban or rural environments– Scientific instruments & specialized tools– Geographic references

Home Versus School

• Division is arbitrary

• Above average score– Probably has extensive knowledge from both

environments

• Below average score– Lack of English vocabulary from home

environment– Not profiting from educational environment

Vocabulary by Grammatical Form

• Part of speech– Usage of word within the test– Noun, verb, or attribute (adjective or adverb)

• Most early items are nouns– Focus on labeling

• Actions & attributes– Verbs, adjectives, adverbs– More abstract than nouns– Classified as “Home” since typically used in

conversation

Crossover Sampling

• Receptive & expressive common words– PPVT-4 Form A & EVT-2 Form B

• 57 possible items– PPVT-4 Form B & EVT-2 Form A

• 59 possible items

• EVT-2 items are open ended– Labeling items: PPVT-4 item listed as common to

EVT-2 item only if the PPVT-4 stimulus word was the most frequent correct EVT-2 response (90%)

– Synonym items: PPVT-4 item listed as common to EVT-2 item only if the PPVT-4 stimulus word was at least 70% of the correct response to the EVT-2 item

Three-tier Model

Tier 1

Tier 2

Tier 3

High-frequency, high-utility

Low-frequency words; technical

words

Known, common

words

Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York: The Guilford Press.

• According to Farber (1999), many high school students develop reading skills through a sixth-grade level but “haven’t actually read enough to develop the vocabulary or general knowledge that more advanced reading requires” (p. 1)– Children must have vocabulary to learn to

read and comprehend, but then they gain vocabulary by continued reading

• Good readers, as described by Lyon (1997), “bring strong vocabularies and good syntactic and grammatical skills to the reading comprehension process…” (p.5)

• Poor readers– Lack of a strategy to learn new vocabulary– May exhibit an overdependence on context– Will expend too much effort to read each word– May use only partial knowledge to determine

meaning

• Provide language-rich activities involving listening and speaking–Read or tell stories

• Ask questions• Ask for a different word• Ask examinee to retell story

–Tell jokes or tall tales• Why is it funny? Unusual? Odd?• Retell changing a key word

• Interactive Word Walls• Brainstorm words that meet criteria (self-selection)• Provide contexts for meanings• Make associations from own existing knowledge and

experiences

• Apply words to real situations– Get children to think more metcognitively about their own

vocabulary and the depth of their understanding about particular words• Have I heard the word used this way?• Have I seen the word used this way in print?• Have I used the word in my writing?

• Build new words using prefixes• If centennial means 100 years, what does

bicentennial mean?

• Compound words• What are the two words in the compound word?

– wiretapping, scatterbrain, underline

• What do they mean separately?

• What do they mean when used together?

Questions?

[email protected]