Latent regression models. Where does the probability come from? Why isn’t the model deterministic. Each item tests something unique – We are interested.

Latent regression models

Where does the probability come from?

• Why isn’t the model deterministic.• Each item tests something unique

– We are interested in the average of what the items assess

• Stochastic subject argument• Random sampling of subjects

Two different models

Pr ; |ni ni i nX x

Pr ; ,ni ni i nX x

Random sampling

Stochastic subject

A Random Effects (Sampling) Model -- 1

2~ ,N

2222

1; , exp22

g

Part 1: A Model for the population

equivalently

A Random Effects Model -- 2

Part 2: A Model for the item response mechanism

Pr ; | ; |

exp1 exp

ni ni i ni i

ni i

i

X x f x

x

Example: SLM (but can be any)

A Random Effects Model -- 3

Part 3: Putting them together

2 2; , , ; | ; ,ni i ni if x f x g d

The unknowns are: 2, iand is not an unknown, it is variable of integration

What is analysed is item response, what is estimated is itemparameters and population parameters

Why do this?

• Solves some theoretical estimation problems– For non-Rasch models

• Provides better estimates of population characteristics

Problems with point estimates

ˆvar var

var var

var

n n n

n n

n

e

e

Problem with discreteness

• For a 6-item test, there are only 7 possible ability estimates to assign to people, those getting a score of 0,1,2,3,4,5,6. (raw score is sufficient statistic for ability)

• Suppose we want to know where the 25th percentile point is. That is, 25% of the population are below this point. We need extrapolation.

The Resulting JML Ability Distribution

Score 0

Score 1

Score 2

Score 3Score 4

Score 5

Score 6

Proficiency on Logit Scale

Distribution for a six item test

Score 0

Score 1

Score 2

Score 3Score 4

Score 5

Score 6

Proficiency on Logit Scale

Traditional approach is a two-step analyses

First estimate abilities

Then compute population estimates such as mean, variance, percentiles using

leads to biased results due to measurement error.

In the case of the population variance, we can correct the bias (disattenuate) by multiplying by the reliability.

But in other cases, it is less obvious how to correct for the bias caused by measurement error.

n̂

n̂

Distribution of Estimates is Discrete

• One ability for each raw score• Ability estimates have a discrete distribution• We imagine (and the model’s premise) is a

continuous distribution• The distribution of ability estimates is

distorted by measurement error

Solutions

• Direct estimation of population parameters (directly via item responses, and not through the estimated abilities)

• Complicated analyses that take into account the error– Not always possible

MML: How it works — 1

• Item Response Model for item i:

• Population Model (discrete)

i

iixf

exp1

exp/1

g()-1.5 0.1-0.5 0.2

0 0.40.5 0.21.5 0.1

MML: How it works — 2

1 1/ 1.5 1.5

1/ 0.5 0.5

1/ 1.5 1.5

1/

/

i i

i

i

i

f x f x g

f x g

f x g

f x g

f x g d continuous case

The Implications — 1

• If , then contains parameters – Note that no ability parameters are involved, only

population parameters.• Use maximum likelihood estimation method

to estimate the item difficulty parameters and population parameters.

• Thus, we directly estimate population parameters through the item responses

2,~ Ng xf

,,,,, 21 I

Bayes Theorem

PrPr |

Pr

Pr Pr | Pr

A BA B

B

A B A B B

PrPr |

Pr

Pr Pr | Pr

A BB A

A

A B B A A

Pr | Pr Pr | Pr

Pr | PrPr |

Pr

A B B B A A

B A AA B

B

The Idea of Posterior Distribution

• If a student’s item response pattern is x then the posterior distribution is given by

Pr | PrPr |

PrB A A

A BB

| ||

|

f g f gh

f f g d

x x

xx x

Pr | PrPr |

Pr

x

xx

The Idea of Posterior Distribution

• Instead of obtaining a point estimate for ability, there is now a (posterior) probability distribution

• incorporates measurement error for

the uncertainty in the estimate.

n̂

|h x

|h x

The Resulting JML Ability Distribution

Score 0

Score 1

Score 2

Score 3Score 4

Score 5

Score 6

Proficiency on Logit Scale

Resulting MML Posterior Distributions

Score 0

Score 1

Score 2

Score 3 Score 4

Score 5

Score 6

Proficiency on Logit Scale

MML EAP Estimates – an aside

Score 0

Score 1

Score 2

Score 3 Score 4

Score 5

Score 6

Proficiency on Logit Scale

MML EAP Estimates – an aside

• Biased at the individual level• Discrete scale, bias & measurement error

leads to bias in population parameter estimates

• Requires assumptions about the distribution of proficiency in the population

Distribution for a six item test

Score 0

Score 1

Score 2

Score 3Score 4

Score 5

Score 6

Proficiency on Logit Scale

Estimating proportions below a point based up posterior distributions

More General Form of the Model

expPr ; , , ,

expn

n nn n n

n

z

x b AξX x b A ξ

z b Aξ

2

2

~ ,

~ ,

N x y z

N

Y β

Item response model

Population model

The underlying population distribution is a mixture of two normal distributions, with different means ( and ).

Population Not Normal

• E.g., sample consists of grades 5 and 8.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69

1 2

• where x=0 if a student is in group 1 and x=1 if a student is in group 2. In this case, we estimate , and . Note that is the difference between the means of the two distributions. That is, group 1 has mean (as x=0), and group 2 has mean (as x=1).

Latent Regression - 1

2,~ xNg

2

Latent Regression - 2

• We call “x” a “regressor”, or a “conditioning variable”, or a “background variable”. We can generalise to include many conditioning variables.

2,~ zyxNg