Monitoring and updating MoM values
Updating MoM values • When to make changes
• Factor updates
• Discussion
CUSUM charts • Practical session
Confidence intervals • Practical session
CUSUM Charts
Practical session: Constructing CUSUM charts
• Quick demonstration in XL
• Some illustrative examples with no random variation to illustrate the methodology
• Discussion of features of the CUSUM charts
Practical session: Constructing CUSUM charts
For each series of MoM values
1. Compute the log10MoM values
2. Produce the CUSUM
3. Plot the CUSUM and interpret
Observation Series 1 Series 2 Series 3 Series 4 Series 5
1 1.1 0.9 1.0 0.9 1.00
2 1.1 0.9 1.0 0.92 1.00
3 1.1 0.9 1.0 0.94 1.00
4 1.1 0.9 1.0 0.96 1.00
5 1.1 0.9 1.0 0.98 1.00
6 1.1 0.9 1.2 1.00 0.99
7 1.1 0.9 1.2 1.02 0.98
8 1.1 0.9 1.2 1.04 0.97
9 1.1 0.9 1.2 1.06 0.96
10 1.1 0.9 1.2 1.08 0.95
11 1.1 0.9 1.2 1.10 0.94
12 1.1 0.9 1.2 1.12 0.93
13 1.1 0.9 1.2 1.14 0.92
14 1.1 0.9 1.2 1.16 0.91
15 1.1 0.9 1.2 1.18 0.9
16 1.1 0.9 1.0 1.2 0.89
17 1.1 0.9 1.0 1.22 0.88
18 1.1 0.9 1.0 1.24 0.87
19 1.1 0.9 1.0 1.26 0.86
20 1.1 0.9 1.0 1.28 0.85
0 5000 10000 15000 20000 25000
-600
-400
-200
0
Sample
log
10 M
oM
Jan
Feb Mar
Apr
May
Jun
Jul
Aug
1 MoM
0.9 MoM
1.1 MoM
0.95 MoM
1.05 MoM
Log MoM CUSUM: Perfect
The horizontal trajectory reflects the situation where there is no bias in MoM values.
0 1000 2000 3000 4000 5000 6000 7000
-150
-100
-50
0
50
100
Sample
log
10 M
oM
Dec
JanFeb M
arApr
May
Jun
Jul
Aug
1 MoM
0.9 MoM
1.1 MoM
0.95 MoM
1.05 MoM
Log MoM CUSUM with bias
Positive bias apparent from May.
Interpretation of CUSUM charts
• CUSUM charts are a powerful way of monitoring MoM values over time
• Interpretation of CUSUM charts can be far from straightforward
• There is always a danger of over interpretation
• Confidence intervals can be used to account for the uncertainty in estimation of median MoM values
Confidence Intervals for Median MoM values
Confidence Intervals
Given a series of n MoM values, the process of obtaining a confidence interval is as follows
1. First ensure that no updates to medians have taken place in the series
2. Second check that the CUSUM scatters about a straight line
3. Compute the log10 MoM values
4. Compute the mean (i.e. average) and standard deviation
5. Compute the standard error (SE = standard deviation/sqrt(n))
6. The lower and upper limits are given by mean ± 1.96×SE
Practical session: Constructing Confidence Intervals
• Quick demonstration in XL
• Hands on
Confidence intervals for MoM values - practical.xls
Updating MoM values
Dealing with changes
• Detecting changes
• When to make changes
• What changes to make
12
• The aim is to keep median MoM levels close to the target by making changes to remove biases that are going to continue
• We need to avoid ‘chasing noise’ which will make things worse
13
Deciding when to make changes
Given an apparent shift in
a temporal plot, try to
identify the cause. Decide
whether that cause is likely
to persist. If so, consider
making a change.
Information on a ‘special
cause’ such as a lot
change would provide
further evidence that the
change is likely to persist.
Deciding when to make changes
log(M
oM
)
14
Lot change
Factor Updates
• If the median MoM is running at a value different from 1 and this situation is going to persist into the future, then make a factor update.
• The methodology is dependent on the software.
ViewPoint
Changes to ViewPoint involve completing a form specifying the median MoM.
ViewPoint
Changes to ViewPoint involve completing a form specifying the median MoM.
LifeCycle
• If a log10 polynomial regression is used for gestational age, the intercept parameter A in the regression is updated by adding log10(Median MoM).
• New A = Current A + log10(Median MoM)
• For example if the median MoM is running consistently at 0.95 log10(0.95) = -0.022276 is added to A.
• New A = Current A - 0.022276
Testing
• It is important to test all updates.
• If the current median MoM is 0.95 and a factor update is applied, then the effect should be to divide by 0.95 so that, for example, a MoM that was 0.95 before the change should become 0.95/0.95 = 1.
• A simple way of testing is to look take samples before the change and verify that after the change the factor has been applied correctly.
Updating the full equations
• Anything other than a factor updates requires specialist input
• This requires • new coefficients
• test data
• Diagnostics based on the data provided
20
21
Discussion
• What processes do you have in your laboratory for applying factor updates?
• How do you document these updates?
• Does the software supplier provide any advice on updating and testing the software configuration?
• Are there any suggestions for improvement?
Laboratory throughput
Rationale
• Need for sufficient precision to estimate performance (standardised SPR)
• To enable monitoring of laboratory medians and early detection of change points
• For proficiency/expertise
log
(M
oM
)
-50
-40
-30
-20
-10
0
10
20
log
(M
oM
)
-50
-40
-30
-20
-10
0
10
20
log
(M
oM
)
-50
-40
-30
-20
-10
0
10
20
log
(M
oM
)
-50
-40
-30
-20
-10
0
10
20
Sample
log
(M
oM
)
-50
-40
-30
-20
-10
0
10
20
0 200 400 600 800 1000
Sample
log
(MoM
)
-250
-200
-150
-100
-50
0
50
0 2500 5000
5 labs 1,000 samples
1 lab5,000 samples
