FITTING PSYCHOMETRIC FUNCTIONS Florian Raudies 11/17/2011 Boston University 1
Dec 16, 2015
1
FITTING PSYCHOMETRIC FUNCTIONSFlorian Raudies
11/17/2011
Boston University
2
Overview
Definitions
Parameters
Fitting
Example: Visual Motion
Goodness of Fit
Conclusion
3
Definitions
Labels for the axes of a psychometric function
Stimulus level
Pro
port
ion
corr
ect
ExamplesExperiment Design
Proportion Correct
2AFC 50…100%
3AFC 33…100%
2IAFC 50…100%
2AFC Two alternative forced choice
4
Definitions
Special points of an psychometric function
Stimulus level
Pro
port
ion
corr
ect
50%
PSE
75%
25%
PSEPoint of subjective equivalence
JNDJust noticeable difference
2JND
Psychometric function
5
Definitions
Weibull function
Cumulative normal distribution function
Logit function
𝑓 𝑊𝑒𝑖𝑏𝑢𝑙𝑙 (𝑥 |𝛼 , 𝛽 )=1− exp (−( 𝑥𝛼 )𝛽
)
𝑓 𝑐𝑛𝑑𝑓 (𝑥 | μ ,𝜎 )=erf (𝑥 ;𝜇 ,𝜎 )
𝑓 𝑙𝑜𝑔𝑖𝑡 (𝑥 | μ ,𝜃 )= 1
1+exp (− 𝑥−𝜇𝜃 )
erf (𝑥 ;𝜇 ,𝜎 )= 1√2𝜋 𝜎 ∫
−∞
𝑥
exp (−(𝑠−𝜇)2
2𝜎)𝑑𝑠
6
Definitions in Matlabfunction Y = weibullFunction(X, alpha, beta)% weibullFunction% X - Input values.% alpha - Parameter for scale.% beta - Parameter for shape.%% RETURN% Y - Return values.%% DESCRIPTION% See http://en.wikipedia.org/wiki/Weibull_distribution.
Y = 1 - exp(-(X/alpha).^beta);
7
Definitions in Matlabfunction Y = cndFunction(X, mu,sigma)% cndFunction - Cumulative normal distribution function% Shift by one up and rescale because the integral for erf % ranges from 0 to value whereas the distribution uses the % boundaries -inf to value.Y = (1+erf( (X-mu)/(sqrt(2)*sigma) ))/2;
function Y = logitFunction(X, mu,theta)% logitFunction…Y = 1./( 1 + exp( -(X-mu)/theta ) );
8
Parameters
0 2 4 6 8 100
0.2
0.4
0.6
0.8
1
stimulus level
prop
ortio
n co
rrec
tPsychometric Functions
Weibull, =5, =7cndf, =7, =1logit, =2, =0.5
9
Parameters
0 1 2 3 4 5 6 7 8 9 100
0.5
1
Stimulus level
Pro
port
ion
corr
ect Weibull
=5, =2=5, =7=1, =7
0 1 2 3 4 5 6 7 8 9 100
0.5
1
Stimulus level
Pro
port
ion
corr
ect Cndf
=5, =1=5, =0.25=1, =0.25
0 1 2 3 4 5 6 7 8 9 100
0.5
1
Stimulus level
Pro
port
ion
corr
ect Logit
=5, =1=5, =0.25=1, =0.25
10
Parameters
Additional parameters for a psychometric function
with the parameter vector .
- scale
- shape
- guessing rate. For nAFC .
- miss rate. For a perfect observer .
11
Definition in Matlabfunction Y = psycFunctionMissGuess(psycFunction,X,Theta,Const)% psycFunctionMissGuess% psycFunction - Function handle for the psychometric function.% X - Input values.% Theta - Parameter values.% Const - Constants, here guess rate and miss rate.%% RETURN% Y - Output values.
Y = Const(1) … + (1 - Const(1) - Const(2)) * psycFunction(X,Theta(1),Theta(2));
12
Fitting
Assume: independent measurements with
strength of the percept and
response of participant in a 2AFC task.
Problem: Maximum likelihood estimation for parameters of the psychometric function
with
13
Fitting
Small values and log-likelihood
The term can lead to small values below the range of single or double precision. Thus, for optimization take the negative and apply the monotonic log-function function:
This expression is maximized for the parameters . Often additional constraints for the parameters are available.
This is a constraint nonlinear optimization problem with also referred to as nonlinear programming.
14
Fitting in Matlabfunction Theta = fittingPsycFunction(X, Y, opt)% fittingPsycFunction…ThetaMin = opt.ThetaMin; % Lower boundary for parameters.ThetaMax = opt.ThetaMax; % Upper boundary.ThetaIni = opt.ThetaIni; % Initial value for parameters.Const = opt.Const; % Constants in the psychometric function.psycFunction = opt.psycFunction; % Function handle for the psychometric function.% Optimization with the fmincon from the Matlab optimization toolbox.Theta = fmincon(@(Theta)logLikelihoodPsycFunction(... psycFunction, Theta, X,Y, Const), ... ThetaIni, [],[],[],[], ThetaMin,ThetaMax, [], opt); function L = logLikelihoodPsycFunction(psycFunction, Theta, X,Y, Const)% logLikelihoodFunction…Xtrue = X(Y==1);Xfalse = X(Y==0);L = -sum(log( psycFunctionMissGuess(...
psycFunction, Xtrue, Theta, Const) + eps))... -sum(log(1 - psycFunctionMissGuess(...
psycFunction, Xfalse, Theta, Const) + eps));
15
Example: Visual Motion
Objective: Measure the coherence threshold for motion-direction discrimination.
Design: 2AFC task between leftward and rightward motion for varying motion coherence by a
limited dot lifetime in an random dot kinematogram (RDK).
Use the method of constant stimuli for 11 coherence values. This requires usually more samples than adaptive thresholding techniques.
Use 10 trials for each coherence value.
This is a very simplified example!
16
Example: Visual Motion
17
Example: Visual Motion
A single trial
Fixation &
(Response) for500ms
Fixation &
1st Motion for400ms
Fixation &
Blank for100ms
Time
Overall time 10 x 11 x 1,9sec = 209sec or 3.48min.
A response is not expected before the first trail.
Are the motions equal?
Fixation&
2nd Motion for400ms
18
Example: Visual MotionCorrect / response
Motion coherence (%)
0 10 20 30 40 50 60 70 80 90 100
Trial
1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1 0 0 0
2 0/1 0/1 0/0 1/1 0/0 1/1 1/1 1 1 1 0
3 1/0 0/1 1/1 0/1 1/1 1/0 0/1 0 0 0 1
4 1/1 1/0 0/1 0/0 0/1 0/1 0/0 0 0 1 1
5 0/0 1/1 1/0 1/0 1/0 0/0 1/1 1 1 1 0
6 0/1 0/0 1/0 1/1 1/1 0/0 1/1 0 1 1 0
7 1/0 1/1 0/0 1/0 0/1 1/1 1/1 1 0 0 1
8 0/0 0/1 0/1 0/1 0/0 0/0 0/0 0 0 0 0
9 1/1 0/0 1/0 0/1 1/1 1/0 0/0 1 1 1 1
10 0/1 1/0 0/0 0/0 0/1 0/0 0/0 0 1 0 0
Correct (%)
50 50 50 50 60 70 90 100 100 100 100
19
Example: Visual Motion
0 20 40 60 80 10040
50
60
70
80
90
100
110
motion coherence (%)
perc
enta
ge c
orre
ctFitted Weibull function
datafitted Weibull
20
Example in Matlab% Load data files.DataStimulus = dlmread('./DataMotionCoherenceStimulus00.txt');DataObserver = dlmread('./DataMotionCoherenceObserver00.txt');% "Response = 1" encodes correct and "Response = 0" incorrect.Response = double(DataStimulus(2:end,:)==DataObserver(2:end,:));trialNum = size(Response,2);StimulusLevel = DataStimulus(1,:);
% Fit data.opt.ThetaMin = [ 1.0 0.5]; % alpha, beta to optimize.opt.ThetaMax = [100.0 10.0];opt.ThetaIni = [ 5.0 1.0];opt.Const = [ 0.5 0.0]; % gamma, lambda are fixed.opt.psycFunction = @weibullFunction;StimulusLevelMatrix = repmat(StimulusLevel,[trialNum 1]);Theta = fittingPsycFunction(StimulusLevelMatrix, Response, opt);
21
Goodness of Fit
Over dispersion or lack of fit
Dependency between trials
Non-stationary psychometric function (e.g. learning)
Under dispersion or fit is too god
Experimenter’s bias in removing outliers
22
Conclusion
Use maximum likelihood to fit your data, while leaving the lapse rate as parameter being optimized.
This is not the case in the presented code but can be adapted.
Assess goodness of fit to:
Ensure Parameter estimates and their variability are from a plausible model to describe the data.
Identify uneven sampling of the stimulus level or outliers by applying an objective criteria.
23
References
For fitting data
Myung, Journal of Mathematical Psychology 47, 2003
Treutwein & Strasburger, Perception & Psychophysics 61(1), 1999
For goodness of fit
Wichmann & Hill, Perception & Psychophysics 63(8), 2001
For detection theory
Macmillan & Creelman. Detection theory - A user’s guide Psychology Press (2009)