Outline Introduction Formulating optimal decision making process. Utility of Bayesian Model Understanding Human Sequential Decision Making Under Uncertainty Using Ideal Observer Analysis Brian J. Stankiewicz, Ph.D. University of Texas, Austin Department Of Psychology & Center for Perceptual Systems June 17, 2005 Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
135
Embed
Understanding Human Sequential Decision Making Under ......Understanding Human Sequential Decision Making Under Uncertainty Using Ideal Observer Analysis Brian J. Stankiewicz, Ph.D.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Understanding Human Sequential Decision MakingUnder Uncertainty Using Ideal Observer Analysis
Brian J. Stankiewicz, Ph.D.
University of Texas, AustinDepartment Of Psychology & Center for Perceptual Systems
June 17, 2005
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Collaborators
University of Texas, Austin
Matthew deBrechtChris GoodsonKyler EastmanMatthew McCabeAnthony Cassandra
University of Minnesota
Gordon E. LeggeErik Schlicht
SUNY Plattsburgh
J. Stephan Mansfield
Army Research Lab
Sam Middlebrooks
University XXI / Army Research Labs
National Institute of Health
Air Force Office of ScientificResearch
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
1 IntroductionFormat of sequential decision making with uncertainty tasks.Problem StructureRe-orientation taskSeek & Destroy problem
2 Formulating optimal decision making process.Tiger Problem
3 Utility of Bayesian ModelNavigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Format of sequential decision making with uncertainty tasks.Problem StructureRe-orientation taskSeek & Destroy problem
Examples Sequential Decision Making with Uncertainty
Medical Diagnosis
Scientific Exploration
Re-orientation after becoming lost
Seek & Destroy
Tiger Problem
Etc.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Format of sequential decision making with uncertainty tasks.Problem StructureRe-orientation taskSeek & Destroy problem
Examples Sequential Decision Making with Uncertainty
Medical Diagnosis
Scientific Exploration
Re-orientation after becoming lost
Seek & Destroy
Tiger Problem
Etc.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Format of sequential decision making with uncertainty tasks.Problem StructureRe-orientation taskSeek & Destroy problem
Sequential Decision Making: Structure
Structure of Sequential Decision Making with UncertaintyTasks.
The decision maker’s current belief about the true state of thesystem.Represented as the probability of being in each state.
2 Hidden state (strue)
The position of the TigerNot directly observable
3 Goal
Maximize the expected reward.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1
Listen HearLeft 0.85
2
Listen HearLeft 0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen
HearLeft 0.85
2
Listen HearLeft 0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft
0.85
2
Listen HearLeft 0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2
Listen HearLeft 0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft
0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3
Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight
0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4
Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4 Listen HearLeft
0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4 Listen HearLeft 0.9698
5
Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4 Listen HearLeft 0.9698
5 Listen HearLeft
0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4 Listen HearLeft 0.9698
5 Listen HearLeft 0.9945
But what about action selection?
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Belief Updating
Belief Updating
p(s ′|b, o) =p(o|s ′, b)p(s ′|b))
p(o|b)
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Listen HearRight 0.85
4 Listen HearLeft 0.9698
5 Listen HearLeft 0.9945
But what about action selection?Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Reward Structure for Tiger Prob-
lem
Tiger=Left Tiger=Right
Listen -1 -1
Open-Left -100 10
Open-Right 10 -100
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Computing Expected Value
ρ(b, a) =∑
s∈S R(s, a)b(s)
Immediate reward
R = r(0) +∑∞
t=1 r(t)
Consider immediate and future rewards
V (b) = maxa∈A
[ρ(b, a) +
∑b′∈B τ(b, a, b′)V (b′)
]Expected Value Function
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Computing Expected Value
ρ(b, a) =∑
s∈S R(s, a)b(s)
Immediate reward
R = r(0) +∑∞
t=1 r(t)
Consider immediate and future rewards
V (b) = maxa∈A
[ρ(b, a) +
∑b′∈B τ(b, a, b′)V (b′)
]Expected Value Function
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Computing Expected Value
ρ(b, a) =∑
s∈S R(s, a)b(s)
Immediate reward
R = r(0) +∑∞
t=1 r(t)
Consider immediate and future rewards
V (b) = maxa∈A
[ρ(b, a) +
∑b′∈B τ(b, a, b′)V (b′)
]Expected Value Function
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Expected Reward
ρ(b, a) =∑
s∈S R(s, a)b(s)Immediate reward
-120.0
-100.0
-80.00
-60.00
-40.00
-20.00
0.000
20.00
0 0.2 0.4 0.6 0.8 1
Expected Reward1 Action
ListenOpen-RightOpen-Left
Expe
cted
Val
ue
p(TigerLeft
)
Need to calculate theexpected reward.
Reward(Action,State)
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
R = r(0) +∑∞
t=1 r(t)
Immediate and future rewards
V (b) = maxa∈A
[ρ(b, a) +
∑b′∈B τ(b, a, b′)V (b′)
]Expected Value Function
Expected rewardfunctions for multiplefuture actions with aninfinite horizon.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
R = r(0) +∑∞
t=1 r(t)
Immediate and future rewards
V (b) = maxa∈A
[ρ(b, a) +
∑b′∈B τ(b, a, b′)V (b′)
]Expected Value Function
Expected rewardfunctions for multiplefuture actions with aninfinite horizon.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
From value function(Expected Rewards)we can generate apolicy based upon ourcurrent belief (beliefvector).
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1
Listen HearLeft 0.85
2
Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen
HearLeft 0.85
2
Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft
0.85
2
Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2
Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2
Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft
0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3
Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Open-Right Reward
0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
Tiger Problem: Action Selection
Table: Belief Updating for Tiger Problem
Act. Num Action Observation p(TigerLeft)
0 —- —- 0.5
1 Listen HearLeft 0.85
2 Listen HearLeft 0.9698
3 Open-Right Reward 0.5
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
POMDP: Computing Expected Value
1 Using a POMDP we can generate a policy graph for aSequential Decision Making Under Uncertainty Task.
Policy graph provides us with the optimal action given a beliefabout the true state of the system.
2 Using a POMDP we can compute the Expected Rewardgiven the initial belief state and optimal action selection.
Using the optimal expected reward structure we can comparehuman performance to the optimal performance.By comparing human behavior to the optimal ExpectedReward we can get a measure of efficiency.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Tiger Problem
POMDP: Computing Expected Value
1 Using a POMDP we can generate a policy graph for aSequential Decision Making Under Uncertainty Task.
Policy graph provides us with the optimal action given a beliefabout the true state of the system.
2 Using a POMDP we can compute the Expected Rewardgiven the initial belief state and optimal action selection.
Using the optimal expected reward structure we can comparehuman performance to the optimal performance.By comparing human behavior to the optimal ExpectedReward we can get a measure of efficiency.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Utility of Optimal Observer
What is the utility of computing the optimal solution?
Provides a way to normalize for task difficulty.
For example, Stankiewicz, Legge, Mansfield & Schlicht (UnderReview)
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Utility of Optimal Observer
What is the utility of computing the optimal solution?
Provides a way to normalize for task difficulty.
For example, Stankiewicz, Legge, Mansfield & Schlicht (UnderReview)
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Utility of Optimal Observer
What is the utility of computing the optimal solution?
Provides a way to normalize for task difficulty.
For example, Stankiewicz, Legge, Mansfield & Schlicht (UnderReview)
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Navigating with uncertainty: General
Needed to carefully control the environment (system) so thatwe can provide the optimal navigator (POMDP) with thesame information as the human.
Used virtual reality indoor environments.Environments were randomly generated on a Cartesian grid.Environments were visually sparse.
No object landmarksAble to quantify Observations (p(o|s))Observations are not unique. More than one state cangenerate the same observation (p(o|s) ≤ 1.0)
Specific set of simple actions.
1 Translate forward 1 hallway unit (to be described)2 Rotate right 90◦
3 Rotate left 90◦
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Navigating with uncertainty: Environment
Sample map of environment
Randomly generated environments.
Generated on Cartesian grid.
Translations move from one red square to another.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Navigating with uncertainty: Environment
Sample of navigating through virtual environment
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Procedure
Training:1 Allow free exploration for 3 minutes.
One goal position.Indicated by an auditory signal when subject “walks” over theposition.
2 Have subjects draw environment on cartesian grid.3 Check if map is accurately drawn.4 Return to exploration.5 Repeat until environment is drawn correctly twice in a
row.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Procedure
Testing:
Subject starts from a random state in the environmentSubject’s task is to reach the goal state in the fewest numberof actions possible (minimize cost).
Every action (translation and rotation) has an equal cost.Subjects have to be certain that they are there and indicatethat they are at the goal with a button press.
Actions are deterministic (p(s ′|a, s) = 1.0 or p(s ′|a, s) = 0.0)Observations are deterministic (p(o|s) = 1.0 or p(o|s) = 0.0Measure the number of actions to reach the goal state and becertain that they are there.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Manipulation
Manipulated the number of hallway units that composed theenvironment.
10, 20, 40, and 80 hallway units.
Increasing the layout size increased the memory/processing inthe four areas that we are interested in understanding.
Inefficient processing of observations (O)Inefficient access to their cognitive map (T)Inefficient belief updating (B)Inefficient decision process (POMDP)
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Environments
10 Hallway Environment
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Environments
80 Hallway Environment
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Increasing layout size increasednumber of actions to reach goal state.
Not too surprising . . . need to travelfarther.
Need to control for task difficulty indifferent environments.
Ideal navigator will be sensitive to taskdifficulty.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Increasing layout size increasednumber of actions to reach goal state.
Not too surprising . . . need to travelfarther.
Need to control for task difficulty indifferent environments.
Ideal navigator will be sensitive to taskdifficulty.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Increasing layout size increasednumber of actions to reach goal state.
Not too surprising . . . need to travelfarther.
Need to control for task difficulty indifferent environments.
Ideal navigator will be sensitive to taskdifficulty.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Increasing layout size increasednumber of actions to reach goal state.
Not too surprising . . . need to travelfarther.
Need to control for task difficulty indifferent environments.
Ideal navigator will be sensitive to taskdifficulty.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Increasing layout size increasednumber of actions to reach goal state.
Not too surprising . . . need to travelfarther.
Need to control for task difficulty indifferent environments.
Ideal navigator will be sensitive to taskdifficulty.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Using ideal navigator we can computethe optimal performance.
POMDP provides optimal performancewith no cognitive limitations.
Use performance to get a measure ofefficiency
Efficiency = IdealPerformanceHumanPerformance
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Using ideal navigator we can computethe optimal performance.
POMDP provides optimal performancewith no cognitive limitations.
Use performance to get a measure ofefficiency
Efficiency = IdealPerformanceHumanPerformance
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Using ideal navigator we can computethe optimal performance.
POMDP provides optimal performancewith no cognitive limitations.
Use performance to get a measure ofefficiency
Efficiency = IdealPerformanceHumanPerformance
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Using ideal navigator we can computethe optimal performance.
POMDP provides optimal performancewith no cognitive limitations.
Use performance to get a measure ofefficiency
Efficiency = IdealPerformanceHumanPerformance
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Experiment 1: Results
Effect of layout complexity on navigationperformance
Suggests that the limitingcognitive factor innavigating withuncertainty is aninefficient belief updatingprocess.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Investigated human sequential decision making underuncertainty.
Used Ideal Observer analysis (Bayesian; POMDP) to studycognitive limitations in two different tasks.
Localized cognitive bottleneck in generating, updating andmaintaining an accurate belief vector.
Currently studying if effect generalizes to non-navigationaltask.
Seek & Destroy problemResults suggest belief updating is also the bottleneck.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Investigated human sequential decision making underuncertainty.
Used Ideal Observer analysis (Bayesian; POMDP) to studycognitive limitations in two different tasks.
Localized cognitive bottleneck in generating, updating andmaintaining an accurate belief vector.
Currently studying if effect generalizes to non-navigationaltask.
Seek & Destroy problemResults suggest belief updating is also the bottleneck.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Investigated human sequential decision making underuncertainty.
Used Ideal Observer analysis (Bayesian; POMDP) to studycognitive limitations in two different tasks.
Localized cognitive bottleneck in generating, updating andmaintaining an accurate belief vector.
Currently studying if effect generalizes to non-navigationaltask.
Seek & Destroy problemResults suggest belief updating is also the bottleneck.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Investigated human sequential decision making underuncertainty.
Used Ideal Observer analysis (Bayesian; POMDP) to studycognitive limitations in two different tasks.
Localized cognitive bottleneck in generating, updating andmaintaining an accurate belief vector.
Currently studying if effect generalizes to non-navigationaltask.
Seek & Destroy problemResults suggest belief updating is also the bottleneck.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Investigated human sequential decision making underuncertainty.
Used Ideal Observer analysis (Bayesian; POMDP) to studycognitive limitations in two different tasks.
Localized cognitive bottleneck in generating, updating andmaintaining an accurate belief vector.
Currently studying if effect generalizes to non-navigationaltask.
Seek & Destroy problemResults suggest belief updating is also the bottleneck.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Summary
Interesting note:
Computationally, updating and generating an accurate beliefvector is fast and easy for the computer.However for human, it is difficultSelecting the optimal action given the belief vector is difficultcomputationally.However, data suggests that it is relatively easy for the human.Suggests a symbiotic relationship between computer andhuman.
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty
OutlineIntroduction
Formulating optimal decision making process.Utility of Bayesian Model
Navigation with Uncertainty: IntroductionExperiment 1: Effect of Layout Complexity
Thank you
Thank You
Brian J. Stankiewicz, Ph.D. Human sequential decision making under uncertainty