Fast Inversion of Normal CDF John D. Cook
Jul 03, 2015
Fast Inversion of Normal CDF
John D. Cook
What we’re computing
Basic assumptions
Five or six significant figures is adequate for many applications
Memory is fast and plentiful
Low-level bit operations are faster than arithmetic
Basic Approach
Tabulate values at sample points and use low-order approximations to fill in
Take advantage of binary representation of floating point numbers to decide where to sample
Change variables to avoid arithmetic
Big clever idea
Extract sample points based on the exponent and mantissa
The extraction is extremely fast
It is biased to sample more frequently near zero, exactly where cPhi-1 needs more sampling
IEEE Floating Point Representation (conceptual)
x = +/- 2e m, 1 <= m < 2
Bit 1: sign bit +/-
Bits 2 through 9: exponent e
Bits 10 through 32: mantissa
IEEE Floating Point Representation (details)
Bit 1: 0 for positive, 1 for negative
Bits 2 through 9: exponent of 2 biased by 127 (values 0 through 255 correspond to actual exponents -127 through 128)
Bits 10 through 32: mantissa minus 1(leading bit always 1, so don’t store)
Starting Point for Marsaglia’s algorithm
Represent numbers byu = 2-k (2-1 + 2-6 j + 2-24 m)
0 <= k < 32, 0 <= j < 320 <= m < 224
k = 126 – e, where ‘e’ is the exponent representation bits
j = first five mantissa representation bits
m = last 18 mantissa representation bits
Sample Points
Tabulate cPhi-1 at points corresponding to m = 0, i.e. at 32 possible values of i and j, a total of 1024 points.
Use quadratic Taylor approximation based at these points
A fixed number samples per exponent samples more finely near zero, just where cPhi-1 needs more samples
Clever indexing
Conceptually, we have a matrix A[i][j] of tabulated values
This requires two calculations to find indices –one for i and one for j – and two operations to lookup values
Combine into a single index n = 992-32k + jthat can be extracted directly by one bit manipulation: bits 2 through 14 minus 3040
Polynomial evaluation
Taylor approximation:t = h B(k,j)x = A(k,j) -0.5 t + 0.125 A(k,j)t2
Rescale B’s by square root of 8:t = h B’x = A – c t – A t2 [ c = sqrt(2) ]
Horner’s method:x = A – t(c – A t)
C++ Implementation
double NormalCCDFInverse(double x)
{
float f1 = (float) x;
unsigned int ui;
memcpy(&ui, &f1, 4);
int n = (ui >> 18) - 3008;
ui &= 0xFFFC0000;
float f2;
memcpy(&f2, &ui, 4);
double v = (f1-f2)*B[n];
return A[n] - v*(sqrt2 - A[n]*v);
}
Fine Print
This algorithm only valid for p <= 0.5
For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p)
Phi-1(p) = cPhi-1(1-p)
Algorithm not valid for p < 2-33
The maximum error is 0.000004, which occurs near p = 0.25
Implementation
double PrivateNormalCCDFInverse(double x) …
double NormalCDFInverse(double x) {
return (x > 0.5)
? PrivateNormalCCDFInverse(1.0-x)
: PrivateNormalCCDFInverse(x);
}
double NormalCCDFInverse(double x) {
return (x > 0.5)
? -PrivateNormalCCDFInverse(1.0-x)
: PrivateNormalCCDFInverse(x);
}
References
Rapid evaluation of the inverse of the normal distribution functionby Marsaglia, Zaman, and MarsagliaStatistics and Probability Letters19 (1994) 259 – 266
Notes
This talk presented June 6, 2001
http://www.JohnDCook.com