PASSING & BABLOK AND DEMING REGRESSION
Vanja Radišić Biljak
Department of medical laboratory diagnostics, University Hospital „Sveti Duh”, Zagreb, Croatia
PASSING & BABLOK AND DEMING REGRESSION
BLAND & ALTMAN PLOT
Vanja Radišić Biljak
Department of medical laboratory diagnostics, University Hospital „Sveti Duh”, Zagreb, Croatia
METHOD COMPARISON
Vanja Radišić Biljak
Department of medical laboratory diagnostics, University Hospital „Sveti Duh”, Zagreb, Croatia
Method verification
• Confirming that declared test specifications can be
achieved in the clinical laboratory
• Precision
• Accuracy
• Linearity
• Interferences
• LOD, LOQ...
• Method comparison
When and why?
Introducing new method or analyzer
Multiple analytical systems in laboratory
Using services of another laboratory
Method change is not going to influence
laboratory result for the patient
How?
• Experimental procedures following protocols
• CLSI EP09-A3: Measurement procedure comparison and
bias estimation using patient samples
1. Number of samples
2. Measurement range
3. Time of analysis
4. Data analysis
5. Data interpretation
Number of samples
• Min: 40 samples
• Optimal: 100 samples • To identify unexpected errors from sample matrix or
interferences
• Measurements in duplicate: use mean value
• Measurements in triplicate: use median value
Measurement range
• Cover 90% of the method measurement range
Method A
Method B
Measurement range
Good
agreement
between
methods Difference in
higher
concentration
range
Measurement range
• Overlaping measurement range for both methods
Glucose
concentration
Method A
Method B
Method A using
dilution protocol
Method B
reported as
LOQ
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Analyzing results
• Comparison of two methods for testosterone
concentration measurement
Summary data
nmol/L Method 1
N=122
Method 2
N=122
Analyzer E411
(Roche)
Architect 1000i
(Abbott)
Min-Max 0.297 – 3.34 0.78 – 2.83
Mean ± SD 1.38 ± 0.50 1.43 ± 0.38
Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)
P (normality) <0.001
REJECTED
<0.001
REJECTED
Scatter plot
Analyzing results
• Comparison of two methods for testosterone
concentration measurement
Summary data
nmol/L
Method 1
N=122
Method 2
N=122
Analyzer E411
(Roche)
Architect 1000i
(Abbott)
Min-Max 0.297 – 3.34 0.78 – 2.83
Mean ± SD 1.38 ± 0.50 1.43 ± 0.38
Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)
P (normality) <0.001
REJECTED
<0.001
REJECTED
Types of correlation
• Pearson (rp) and Spearman correlation (rs)
122 samples < 30 Spearman
correlation
Numerical data At least one
ordinal data
Spearman
correlation
Reject normality
Both variables
do not follow
normal
distribution
Spearman
correlation
Pearson correlation
Correlation
• Spearman coefficient of correlation
Coefficient of correlation (r)
Coefficient of
correlation (r) Interpretation
0-0.24 No association
0.25-0.49 Poor association
0.50-0.74 Moderate to good
association
0.75-1.00 Very good to excellent
association
r can be interpreted only if P < level of significance (0.05)
What is the meaning of this result?
• Methods are significantly associated
• Linear relation between methods
• ↑ of Method A associated with ↑ of Method B
• Nothing about interchangeability of
methods!
WHY?
Example 1
r (95% CI) =
1.00 (1.00-1.00)
Method 2 =
Method 1 + 10
Method 2 = Method 1
Example 2
r (95% CI) =
1.00 (1.00-1.00)
Method 3 =
1,5 x Method 1
Method 2 = Method 1
Example 3
r (95% CI) =
1.00 (1.00-1.00)
Method 4 =
1,5 x Method 1 + 20
Method 2 = Method 1
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Analyzing results
• Comparison of two methods for testosterone
concentration measurement
Summary data
nmol/L
Method 1
N=122
Method 2
N=122
Analyzer E411
(Roche)
Architect 1000i
(Abbott)
Min-Max 0.297 – 3.34 0.78 – 2.83
Mean ± SD 1.38 ± 0.50 1.43 ± 0.38
Median (IQR) 1.33 (0.99 – 1.69) 1.38 (1.15 – 1.63)
P (normality) <0.001
REJECTED
<0.001
REJECTED
Significance of difference
• Wilcoxon test (normality failed)
P=0.02
Significance of difference
• Wilcoxon test (normality failed)
P=0.02
Significant
difference between
methods
What is the meaning of this result?
• Calculating differences for each pair of measurement
• Comparing number of negative and positive differences
• If there is no difference between methods, number of
differences is equal
More measurements were higher using Method 2
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Bland-Altman analysis
• Graphical method to compare two measurements
technique
• Analyzing differences between measurement pairs
• Identification of constant and proportional bias
Giavarina D. Understanding Bland Altman analysis. Biochem Med (Zagreb) 2015;25(2):141-51.
Calculating differences
Average = - 0.05
Ideal situation: all differences = 0
Real situation (analytical
variability): all differences
close to 0 (some +; some -)
The average of all calculated
differences between measurement
pairs close to 0
Bland-Altman plot
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
First method or
Reference method
(true value)
Laboratory methods
(no true value)
95% CI
95% CI
-1.96 s
LOA
Bland-Altman plot
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
First method or
Reference method
(true value)
Laboratory methods
(no true value)
95% CI
95% CI
-1.96 s
LOA
Evaluating difference
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
95% CI Including 0
No statistically significant
difference
Evaluating difference
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
95% CI Excluding 0
Statistically significant
difference
Type of difference
Mean difference
Mean of methods
Diffe
rence in m
easure
ment
units
Meth
od 2
– M
eth
od 1
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
Evaluating
constant bias
Type of difference
Mean difference
Mean of methods
Diffe
rence in p
erc
enta
ges
(Me
thod 2
– M
eth
od 1
)/M
eth
od 1
x100%
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
Evaluating
proportional
bias
Limits of agreement
Normal distribution → 95% cases within ± 1.96 s
Limits of agreement
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
95% of the
differences within
LOA
Limits of agreement
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
Narrow LOA
Good agreement
Wide LOA
Poor agreement
Precision of limits of
agreement
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
95% CI
95% CI
-1.96 s
LOA
95% CI of limits of
agreement
Distribution of differences – Mountain plot
Distribution of differences – Mountain plot
P = 0.097
Normality accepted
Distribution of differences – Mountain plot
P = 0.097
Normality accepted
How are differences
distributed
according to
concentration level?
Distribution of differences – Mountain plot
P = 0.097
Normality accepted
How are differences
distributed
according to
concentration level?
Bland-Altman
plot
Testosterone comparison (1)
1) Mean difference -0.05 mmol/L
2) Difference statistically
significant
3) Constant bias
4) Shape of the cloud
indicates aditional
proportional bias
Testosterone comparison (2)
1) Mean difference -0.05 mmol/L
2) Difference statistically
significant
3) Constant bias
4) Shape of the cloud
indicates aditional
proportional bias
Testosterone comparison (3)
1) Mean difference -6.9%
2) Difference statistically
significant
3) Proportional bias
Testosterone comparison (4)
1) Mean difference -6.9%
2) Difference statistically
significant
3) Proportional bias Is this difference clinically significant?
Data interpretation
Statistical significance Clinical significance
Comparing values
with predefined
acceptance
criteria
https://www.westgard.com/biodatabase1.htm
Testosterone comparison (5)
13,6%
allowed
TE
https://biologicalvariation.eu
BA = 0.25 (CVI2 + CVG
2)1/2
Testosterone comparison (6)
Summary data
nmol/L
Method 1
N=122
Method 2
N=122
Reference
method
Analyzer E411 (Roche) Architect 1000i
(Abbott)
LCMS
Min-Max 0.30 – 3.34 0.78 – 2.83 0.54 – 2.43
Mean ± SD 1.38 ± 0.50 1.43 ± 0.38 1.22 ± 0.39
Median (IQR) 1.33
(0.99 – 1.69)
1.38
(1.15 – 1.63)
1.15
(0.94 – 1.42)
P (normality) <0.001
REJECTED
<0.001
REJECTED
<0.001
REJECTED
Bland-Altman plot
Mean difference
Mean of methods
Diffe
rence
0
+1.96 s
LOA
First method or
Reference method
(true value)
Laboratory methods
(no true value)
95% CI
95% CI
-1.96 s
LOA
Testosterone comparison (6)
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Linear regression
High correlation
Linear relationship
Equation to describe relationship
between methods
Determine proportional and constant error
Deming regression
Passing and Bablok regression
Bilić-Zulle L. Comparison of methods: Passing and Bablok regression. Biochem Med (Zagreb) 2011;21:49-52.
Linear regression
Regression equation
y = a + bx
Intercept = a
tg (α) = b
x
y
Method B
α
Method A
95% confidence
intervals y = x
Regression equation
y = a (95% CI) +
b (95% CI) x
Constant and proportional error
Intercept = a
tg (α) = b
x
y
Method B
α
Method A
Regression equation
y = a (95% CI) + b (95% CI) x
Excluding 0
Constant
error
y = x Excluding 1
Proportional
error
Deming regression
• Includes analytical variability of both methods (CV)
• Both methods prone to errors (clinical laboratory)
There are limitations to using Deming regression!!!
Assumes that errors are independent and normally distributed!
Sensitive to outliers!
Testosterone comparison (7)
Testosterone comparison (7)
Testosterone comparison (7)
y = 0.44 (0.36 to 0.52) + 0.72 (0.65 to 0.78) x
Constant bias Proportional difference
Why don’t we recalculate results?
Testo (Architect) = 0.44+ 0.72 x Testo (e411)
Testo (e411) = (Testo (Architect) – 0.44) / 0.72
Residual analysis
• Calculation of the results using the regression equation
• Calculation of the difference between calculated and
measured value
Residual analysis
y
x
Y – F(x)
0
> 0
< 0
All measuremets near the regression line Difference between actual Y and
calculated Y close to 0
Good fit; equation can be used
Residual analysis
x
y
x0
> 0
< 0
Many measuremets far from the regression line Difference between actual Y and
calculated Y different than 0
Poor fit; equation can’t be used
Residual analysis
Differences between measured and calculated values
Passing & Bablok regression
• Non-parametric method
• No assumptions about distributions of samples
• No assumptions about distributions of errors
• Not sensitive to outliers
Confirming linear relation between methods
If linearity test failed, Passing Bablok regression should not be used!
Testosterone comparison (8)
Deming regression
y = 0.44 (0.36 to 0.52) +
0.72 (0.65 to 0.78) x
Passing & Bablok
regression
y = 0.42 (0.35 to 0.50) +
0.73 (0.66 to 0.80) x
Data analysis
Several statistical aproaches:
•Correlation
•Paired test for difference
•Bland-Altman analysis
•Linear regression
• Deming regression
• Passing-Bablok regresion
Some examples…
Summary data
RBC
1012/L
Analyzer 1
N=300
Analyzer 2
N=300
Min-Max 1.82 – 7.15 1.82 – 7.24
Mean ± SD 4.49 ± 0.84 4.56 ± 0.85
Median (IQR) 4.50 (4.07 – 5.01) 4.58 (4.17 – 5.05)
P (normality) 0.01
REJECTED
<0.001
REJECTED
Some examples…
Some examples…
Some examples…
Some examples…
No constant nor
proportional differences
Some examples…
Summary data
HGB
g/L
Analyzer 1
N=300
Analyzer 2
N=300
Min-Max 66 – 243 64 – 245
Mean ± SD 135 ± 27 134.7 ± 26
Median (IQR) 135 (121 – 149) 135 (121.5 – 148)
P (normality) <0.001
REJECTED
<0.001
REJECTED
Some examples…
Some examples…
Some examples…
Some examples…
Some examples…
Some examples…
If linearity test failed, Passing Bablok regression should not be used!
What to do?
CLSI EP09-A3
CLSI EP09-A3
Some examples…
Summary data
PLT
109/L
Analyzer 1
N=300
Analyzer 2
N=300
Min-Max 8 – 1693 8 – 1566
Mean ± SD 248 ± 192 231 ± 178
Median (IQR) 215 (113 – 325) 201 (106 – 302)
P (normality) <0.001
REJECTED
<0.001
REJECTED
Some examples…
Some examples…
Proportional error
0 500 1000 1500 2000
0
200
400
600
800
1000
1200
1400
1600
1800
TRC-S
TR
C-N
Method comparison
• Important laboratory procedure for verification
• Included into validation protocols for new reagents
• Comparison with the reference method
• Comparison with different manufacturers
• Comparison with same manufacturer
• Results are presented in manufacturers declarations
P values missing
95%CI missing
Small number of
samples
P values missing
95%CI missing
P values missing
95%CI missing
P values missing
95%CI missing
P values missing
95%CI missing
THE END