Change Detection in Dynamic Environments Mark Steyvers Scott Brown UC Irvine This work is supported by a grant from the US Air Force Office of Scientific.

Change Detection in Dynamic Environments

Mark Steyvers

Scott Brown

UC Irvine

This work is supported by a grant from the US Air Force Office of Scientific Research (AFOSR grant number FA9550-04-1-0317)

Overview

• Experiments with dynamically changing environments

• Task: Given a sequence of random numbers, predict the next one

• Questions:

– How do observers detect changes?

– What are the individual differences? “Jumpiness”

• Bayesian models + simple process models

= observed data

= prediction

Two-dimensional prediction task

• 11 x 11 button grid• Touch screen monitor• 1500 trials • Self-paced • Same sequence for all

subjects

0

5

10

0

5

10

Subject 4

40 50 60 70 80 90 100 110 120 1300

5

10

Time

Subject 12

Sequence Generation

• (x,y) locations are drawn from two binomial distributions of size 10, and parameters θ

• At every time step, probability 0.1 of changing θ to a new random value in [0,1]

• Example sequence:

Time

θ=.12 θ=.95 θ=.46 θ=.42 θ=.92 θ=.36

Example Sequence

0

5

10

Optimal Bayesian Solution

0

5

10

Subject 4

40 50 60 70 80 90 100 110 120 1300

5

10

Time

Subject 12

= observed sequence

Bayesian Solution= prediction

0

5

10

Optimal Bayesian Solution

0

5

10

Subject 4

40 50 60 70 80 90 100 110 120 1300

5

10

Time

Subject 12

Subject 4 – change detection too slow0

5

10

Optimal Bayesian Solution

0

5

10

Subject 4

40 50 60 70 80 90 100 110 120 1300

5

10

Time

Subject 12

Subject 12 – change detection too fast

(sequence from block 5)

Tradeoffs

• Detecting the change too slowly will result in lower accuracy and less variability in predictions than an optimal observer.

• Detecting the change too quickly will result in false detections, leading to lower accuracy and higher variability in predictions.

0.5 1 1.5 2 2.5 3 3.52.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Mean Absolute Movement

Me

an

Abs

olu

te T

ask

Err

or

12

34

5

6

7

8 9

OPTIMAL SOLUTION

Average Error vs. Movement

= subject

“Ideal” observer: inferring the HMM that generated the data

2x

2

2y

1x

1

1y

tx

t

ty

...

Time 1 2 t t+1

Measurements

states

changepoints

Change probability

1tx

1t

?

Gibbs sampling

Model predictionInferred changepoint

• Sample from distribution over change points. • Prediction is based on average measurement after last inferred

changepoint

0.5 1 1.5 2 2.5 3 3.52.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Mean Absolute Movement

Me

an

Abs

olu

te T

ask

Err

or

12

34

5

6

7

8 9

OPTIMAL SOLUTION

Average Error vs. Movement

= subject

A simple process model

1. Make new prediction some fraction α of the way between recent outcome and old prediction α = change proportion

2. Fraction α is a linear function of the error made on last trial

3. Two free parameters: A, B

A<B bigger jumps with higher error

A=B constant smoothing

1 (1 )t t tp y p

t t tError y p

α

0

1

A

B

BA

Average Error vs. Movement

0.5 1 1.5 2 2.5 3 3.52.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

Mean Absolute Movement

Me

an

Abs

olu

te T

ask

Err

or

12

34

5

6

7

8 9

OPTIMAL SOLUTION

= subject

= model

One-dimensional Prediction Task

1

2

3

4

5

6

7

8

9

10

11

12

12 PossibleLocations

• Where will next blue square arrive on right side?

Average Error vs. Movement

0 0.5 1 1.5 2

1.2

1.4

1.6

1.8

2

2.2

2.4

Mean Absolute Movement

Me

an

Abs

olu

te T

ask

Err

or

12

3

4

56

7

8

910

11

12

13

14

15

16

17

1819

2021

OPTIMAL SOLUTION

= subject

= model

New Experiments

• Prediction judgments might not be best measurement for assessing psychological change

• New experiments:

– Inference judgment: what currently is the state of the system?

Inference Task(aka filtering)

2x

2

2y

1x

1

1y

tx

t

ty

...

Time 1 2 t t+1

1tx

?

1ty

What is the cause of yt+1?

Tomato Cans Experiment

• Cans roll out of pipes A, B, C, or D

• Machine perturbs position of cans (normal noise)

• At every trial, with probability 0.1, change to a new pipe (uniformly chosen)

(real experiment has response buttons and is subject paced)

A B C D

Tomato Cans Experiment

(real experiment has response buttons and is subject paced)

A B C D • Cans roll out of pipes A, B, C, or D

• Machine perturbs position of cans (normal noise)

• At every trial, with probability 0.1, change to a new pipe (uniformly chosen)

• Curtain obscures sequence of pipes

Tasks

A B C D• Inference:

what pipe produced the last can?

A, B, C, or D?

Cans Experiment

• 136 subjects

• 16 blocks of 50 trials

• Vary change probability across blocks

– 0.08

– 0.16

– 0.32

• Question: are subjects sensitive to the number of changes?

50

60

70

80

90

100low alpha

50

60

70

80

90

100

% A

ccu

racy

(ag

ain

st T

rue

)med alpha

0 10 20 30 40 50 6050

60

70

80

90

100

% Changes

high alpha

ideal

ideal

ideal

Prob. = .08

Prob. = .16

Prob. = .32

Plinko/ Quincunx Experiment

Physical version Web version

Conclusion

• Adaptation in non-stationary decisionenvironments

• Individual differences

– Over-reaction: perceiving too much change

– Under-reaction: perceiving too little change

Do the experiments yourself:

http://psiexp.ss.uci.edu

Number of Perceived Changes per Subject

Low medium high

Change Probability

(Red line shows ideal number of changes)

Subject #1

Number of Perceived Changes per Subject

55% of subjects show increasing pattern

45% of subjects show non-increasing pattern

Low, medium, high change probability Red line shows ideal number of changes

Tasks

A B C D• Inference:

what pipe produced the last can?

A, B, C, or D?

• Prediction: in what region will the next can arrive?

1, 2, 3, or 4?

1 2 3 4

Cans Experiment 2

• 63 subjects

• 12 blocks

– 6 blocks of 50 trials for inference task

– 6 blocks of 50 trials for prediction task

– Identical trials for inference and prediction

INFERENCE PREDICTION

Sequence

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

1 2 3 4 5 6A B C D

Trial

INFERENCE PREDICTION

Sequence

Ideal Observer

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

1 2 3 4 5 6A B C D

Trial

INFERENCE PREDICTION

Sequence

Ideal Observer

Individualsubjects

Trial0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

1 2 3 4 5 6A B C D

INFERENCE PREDICTION

Sequence

Ideal Observer

Trial0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

1 2 3 4 5 6A B C D

Individualsubjects

INFERENCE PREDICTION

Sequence

Ideal Observer

Trial0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

1

2

3

4

5

6

1 2 3 4 5 6A B C D

Individualsubjects

0 20 40 6030

40

50

60

70

80

90

100

Changes (%)

Acc

ura

cy

0 20 40 6030

40

50

60

70

80

90

100

Changes (%)

Acc

ura

cy

= Subjectideal

ideal

INFERENCE PREDICTION

0 20 40 6030

40

50

60

70

80

90

100

Changes (%)

Acc

ura

cy

ideal

0 20 40 6030

40

50

60

70

80

90

100

Changes (%)

Acc

ura

cy

ideal

INFERENCE PREDICTION

= Process model

= Subject