J. Elder PSYC 6256 Principles of Neural Coding 2. SIGNAL DETECTION THEORY
J. Elder PSYC 6256 Principles of Neural Coding
2. SIGNAL DETECTION THEORY
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
2
Signal Detection Theory
Provides a method for characterizing human performance in detecting, discriminating and estimating signals.
For noisy signals, provides a method for identifying the optimal detector (the ideal observer) and for expressing human performance relative to this.
Origins in radar detection theory Developed through the 1950s and on by Peterson,
Birdsall, Fox, Tanner, Green & Swets
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
3
Example 1
The observer sits in a dark room On every trial, a dim light will be flashed with 50%
probability. The observer indicates whether she believes the
light was flashed or not. This is a yes-no detection task.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
4
Noise
In this example, the information useful for the task is the light energy of the stimulus.
By the time the stimulus information is received by decision centres in the brain, it will be corrupted by many sources of noise:
photon noise
isomerization noise
neural noise
Many of these noise sources are Poisson in nature: the dispersion increases with the mean.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
5
Equal-Variance Gaussian Case
It is often possible to approximate this noise as Gaussian-distributed, with the same variance for both stimulus conditions.
Then the noise is independent of the signal state.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
6
Discriminability d’
p x |S = sH( ) = 12πσ
exp −x − µH( )2
2σ 2
⎛
⎝⎜⎜
⎞
⎠⎟⎟
p x |S = sL( ) = 12πσ
exp −x − µL( )2
2σ 2
⎛
⎝⎜⎜
⎞
⎠⎟⎟
µH − µL
σ
d ' = signal separation
signal dispersion=µH − µL
σ
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
7
Criterion Threshold The internal response is often approximated as a continuous
variable, called the decision variable.
But to yield an actual decision, this has to be converted to a binary variable (yes/no).
A reasonable way to do this is to define a criterion threshold z:
x ≥ z → ' yes'x < z → 'no'
x
z
x
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
8
Effect of Shifting the Criterion
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
9
How did we calculate these numbers?
p x |S = sH( ) = 12πσ
exp −x − µH( )2
2σ 2
⎛
⎝⎜⎜
⎞
⎠⎟⎟
p x |S = sL( ) = 12πσ
exp −x − µL( )2
2σ 2
⎛
⎝⎜⎜
⎞
⎠⎟⎟
µH − µL
σ
d ' = zFA − zHIT
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
10
What is the right criterion to use? Suppose the observer wants to maximize the expected number
of times they are right.
Then the optimal decision rule is to always select the state s with higher probability for the observed internal response x:
This is the maximum likelihood detector. For the equal-variance case, this means that the criterion is the
average of the two signal levels:
p x | sH( )p x | sL( ) ≥1→ ' yes '
p x | sH( )p x | sL( ) <1→ 'no '
z =
12
µL + µH( ) z
The ‘likelihood ratio test’
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
11
Optimal Performance
The performance of the maximum likelihood observer for this yes/no task is given by
p(correct) = p(HIT) = p(CORRECT REJECT) = erfc − ′d
2 2
⎛⎝⎜
⎞⎠⎟
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
12
Bias
For this optimal decision rule, the different types of errors are balanced: p(FA) = p(MISS)
For observers that use a different criterion, the different types of errors will be unbalanced.
Such observers have lower p(correct) and are said to be biased.
z
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
13
ROC Curves
Suppose the experiment is repeated many times under different instructions.
The first time, the observer is instructed to be extremely stringent in their criterion, only reporting ‘yes’ when they are 100% sure the light was flashed.
On subsequent repetitions, the observer is instructed to gradually relax their criterion.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
14
ROC Curves As the criterion threshold is swept from right to left, p(HIT)
increases, but p(FA) also increases.
The resulting plot of p(HIT) vs p(FA) is called a receiver-operating characteristic (ROC).
′d = 0
Increasing ′d
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
15
ROC Curves
Note that d’ remains fixed as the criterion is varied! Thus d’ is criterion-invariant, and is thus a pure
reflection of the signal-to-noise ratio.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
16
Example 2: Motion Direction Discrimination
Random dot kinematogram Signal dots are either all moving up or all moving down Noise dots are moving in random directions
Britten et al (1992)
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
17
100% Coherence
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
18
30% Coherence
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
19
5% Coherence
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
20
0% Coherence
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
21
The Medial Temporal Area (V5)
www.thebrain.mcgill.ca
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
22
Experimental Details
Signal direction always in preferred or anti-preferred direction for cell.
What kind of task is this? Note that now there is external noise as well as
internal noise. To calculate neural discrimination performance,
assumed neuron paired with identical neuron, tuned to opposite direction of motion.
Behaviour Neuron Anti-Preferred Direction
Preferred Direction
False Alarm Rate
Hit
Rat
e
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
25
Priors
Note that if the probabilities of the two signal states are not equal, the maximum likelihood observer will be suboptimal.
In this case we must make use of the posterior ratio.
p sH | x( )p sL | x( ) ≥1→ ' yes '
p sH | x( )p sL | x( ) <1→ 'no '
Maximum a posteriori (MAP) rule
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
26
MAP Inference
Using Bayes’ rule, we obtain:
Thus we simply scale the likelihoods by the priors.
p sH | x( )p sL | x( ) =
p x | sH( )p sH( )p x | sL( )p sL( )
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
27
Loss and Risk
Maximizing p(correct) is not always the best thing to do.
How would you adjust your criterion if you were A venture capitalist trying to detect the next Google? A pilot looking for obstacles on a runway?
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
28
Loss Function In general, different types of correct decision or action will
yield different payoffs, and different types of errors will yield different costs.
These differences can be accounted for through a loss function:
Let a(x) represent the action of the observer, given internal response x.
Then L s,a(x)( ) represents the cost of taking action a, given world state s.
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
29
The Ideal Observer
The Ideal Observer uses the decision rule that minimizes the Expected Loss, aka the Risk R(a|x):
R(a | x) = L s,a(x)( )p(s,x)
s∑ = L s,a(x)( )p(x | s)
s∑ p(s)
Probability & Bayesian Inference
J. Elder PSYC 6256 Principles of Neural Coding
30
Example 3: Slant Estimation