Test Analysis Report Slides

7/27/2019 Test Analysis Report Slides

1/18

Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Test Analysis

ReportDetailed information regarding the statistical

performance of an test


2/18

Slide 2 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Overview Detailed information regarding the statistical

performance of an test Report Sections

First section presents information in a tabular format

Stats including mean, standard deviation and reliability

Second section: Distribution of tests scores using afrequency distribution table

lists all scores obtained by examinees

histogram graphically displays the range of scores


3/18


Test Analysis Report First section presents information in a tabular format

Stats including mean, standard deviation and reliability


4/18Slide 4 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Test Analysis Report Second section: Distribution of tests scores using a

frequency distribution table lists all scores obtained by examinees

histogram graphically displays the range of scores


5/18Slide 5 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Report Management and Distribution Distribute via Browser, PDF, Word, Excel, Email,

or Print My Reports


6/18Slide 6Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.

Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

Internal Consistency

Questionmark uses Cronbachs Alpha TestReliability

Reason its the gold standard:

Highly documented and very well researched (60+years)

Can be used with a wide variety of question types(partial credit and right/wrong)

Most widely used method in high and medium stakes


7/18Slide 7 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.


Cronbachs Alpha Internal consistency test

reliability: A measure of how

well the assessment questionsrelate to one another tomeasure the same topic

Crucial information for legaldefensibility is the testreliablymeasuring what is wasdesigned to measure?

Speaks to the quality ofassessment questions: Are theyof high statistical quality?

)(

1 2

22

T

pT

S

SS

k

k




Other Measures Skew

Mode

Kurtosis

Mean (arithmetic)

Median

Mode

Standard Deviation

Variance

Standard Error of measurement

Standard Error of the Mean




SkewA measure of the symmetry of the distribution

of scores are scores are pushed or skewed to one side or

the other?

Ranges from about -2 to +2

0

2

4

6

8

10

12

14

16

18

20

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

Participant scores

Positive skew

0

2

4

6

8

10

12

14

16

18

20

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

Participant scor es

Negative s kew

0

2

4

6

8

10

12

14

16

18

20

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0% 20% 40% 60% 80% 100%

Participant scores

Normal distribution (no skew)


10/18


Kurtosis A measure of the symmetry of a distribution of

scores How peaked/pointed versus flat are the distribution of

scores? What is happening at the tails?

Normal range from about -3 to +3

0

2

4

6

8

10

12

14

16

18

20

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

Participant sc ores

Normal distribution (zero kurtosis: mesokurtic)

0

2

4

6

8

10

12

14

16

18

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

Participant sco res

Negative kurtosis (flat: platykurtic)

0

2

4

6

8

10

12

14

16

18

20

#

o

f

p

a

r

ti

c

i

p

a

n

t

s

0 % 1 0% 2 0 % 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

Participant scor es

Positive kurtosis (pointed: leptokurtic)

( )

)3)(2(

)1(3

)3)(2)(1(

)1( 24

+

nn

nz

nnn

nn


11/18


Mean (arithmetic) The most commonly used measure of central tendency

(refers to the middle of a distribution of scores)

Range of values depends on scores


12/18


Median Another measure of central tendency, less sensitive than

the mean to outliers

Range of values depends on scores Where 50% of participants obtained higher scores and

50% of participants obtained lower scores


13/18


Mode A third measure of central tendency, used a great deal in

survey analysis The most common score in a distribution of scores


34% 43% 56% 56% 56% 63% 67% 76% 88%

Mode = 56%


14/18


Standard deviation The spread or variation of scores between participants

Are the scores spread out (e.g., 0 to 100%) or clusteredtogether (e.g., all scores between 55 and 62%)


Ricks testscore = 75%

Sallys testscore = 83%

Marks testscore = 53%

Ellas testscore = 91%

Standard deviation = 16.36%


15/18


VarianceAnother measure of variation

The first step in calculating a standarddeviation: The standard deviation is thesquare root of the variance


Used in some advanced calculations (e.g.,

Analysis of variance: ANOVA, Multiple analysisof variance: MANOVA)


16/18


Standard Error of

measurement

Theoretical test score = 66.4%





Product knowledge test

Theoreticalstandard

deviation =1.26%

1.26% oferror

surroundingRicks observed

scoreRicks observedscore = 66.1%

observed test score = theoretical true score + error


17/18


Standard Error of the Mean

0

5

10

15

20

25

30

35

#

o

f

p

a

rt

i

c

i

p

a

n

t

s

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Participant score s

Sample (153 participants)

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

#

o

f

p

a

rt

i

c

i

p

a

n

t

s

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Participant score s

Population (87,000 participants)

Sample mean = 56.78%

Sample standard deviation = 15.21%Standard error of mean = 1.23%

___________________________within plus or minus 1 standard error

of the mean value, 68 times out of 100,the true population mean will reside

Typically the population informationis not known, but if you could see

the information

Population true mean = 57. 81%


18/18

Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

More information:

www.questionmark.com

Test Analysis Report Slides

Documents