Statistical Analysis of IC50s Nick Andrews, Statistics Unit CFI, HPA.

Statistical Analysisof IC50s

Nick Andrews, Statistics Unit CFI, HPA

Outline

• Curve fitting to estimate IC50

• Determining cut-offs for outliers (high IC50)

• Monitoring trends in IC50

Curve fitting

The data in each run consists of measuring the RFU of a sample against the antiviral at different concentrations (ten 4-fold dilutions) as well as with no-antiviral (virus-control)

Estimate IC50: the concentration at which the rfu is 50% of that from the virus control.

Sources of variation in IC50

• Curve fitting

• Within a run (2 – replicates)

• Between runs (Reference samples)

Point to Point (Excel)

Oseltamivir- Plate 1

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

40

00

10

00

25

0

62

.50

15

.62

50

3.9

06

3

0.9

76

6

0.2

44

1

0.0

61

0

0.0

15

3

VC

Oseltamivir Conc (nM)

RF

U

292R 292K 119V A/Lisbon/22/2007

Smooth Curves (S-shaped Graph-pad)

Comparison-curve fitting

Sample P-P S-shape Ratio (Hi/Low)292R 0.69 0.69 1.00292K >4000 8551 -119V 40.3 40.7 1.01A/Lisbon22/22070.59 0.56 1.05

A/Latvia/685/2007 0.68 0.69 1.01A/Denmark/1/2007 0.43 0.41 1.05A/Denmark/2/2007 0.54 0.51 1.06A/Denmark/3/2007 0.54 0.48 1.12

Comments (Curve fitting)

• For most samples curve will make no more than about 5% difference to results.

• Must be clear exactly how the curve fitting and calculation of IC50 works – e.g. is IC50 based on 50% of OD of virus control or 50% of fitted upper asymptote.

• Need to ensure problems in curve fitting are flagged.

Between replicate variation (Point to point)

Sample Rep1 Rep2 Ratio(High/Low)

292R 0.75 0.63 1.19292K 3769 >4000 -119V 42.0 38.6 1.09A/Lisbon22/22070.57 0.61 1.07A/Latvia/685/2007 0.70 0.66 1.06A/Denmark/1/2007 0.40 0.46 1.15A/Denmark/2/2007 0.58 0.49 1.18A/Denmark/3/2007 0.54 0.53 1.02

Comments on replicates

• Between replicates in a run variation is greater than curve fitting with most results within about 15%-20%.

• Using replicates and taking the average reduces this effect.

• Large differences between replicates should be flagged (e.g. >30%).

Between run variation

Comparison runs (Reference Virus)

Sample Run1 Run2 Ratio(High/Low)

274H Osel 0.42 0.36 1.17274H Zan 0.17 0.22 1.29274Y Osel 300 303 1.01274Y Zan 0.31 0.37 1.19119V Osel 43.7 34.0 1.29199V Zan 1.41 1.32 1.07DB Osel 920 1316 1.43DB Zan 239 351 1.47

Comments

• Between runs variation is higher still at about 40%+. Hence retest those with high values and use controls in runs to identify problems in a run.

Determining Cut-offs

• The aim is to pick up outliers with high IC50 results as they may indicate resistant strains.

• Need a method to determine the ‘normal range’ in the absence of outliers.

• Various statistical methods may be used – the important thing is to ensure the outliers do not unduly affect the cut-off.

Methods used

• Need sufficient data points for initial estimate (about 30-50). These can be refined later with more data.

• Advisable to transform data to approximate normality (log-transform).

• Obtain a robust estimation of the Standard deviation (1.48*Median absolute deviation or using 0.75*Inter-quartile range).

• Use median + say 1.65, 3 SD.

Example: Initial 30 results

H3N2 Zanamivir 2006/7

0.10

1.00

10.00

0 30 60 90 120 150 180

Virus

IC50

Median=0.47 Robust SD (log-scale) = 0.17

Cut1 =1.07 Cut2=1.81

Example: Final results 2006/07

H3N2 Zanamivir 2006/7

0.10

1.00

10.00

0 30 60 90 120 150 180Virus

IC50

Median=0.48 Robust SD (log-scale) = 0.12

Cut1 =0.93 Cut2=1.35

Do we need to log-transform?

• Most results from dilution assays produce ‘geometric results’ so likely to be sensible

• Sometimes doesn’t seem to matter but sometimes data are skewed. (e.g. lower quartile much closer to median than upper quartile) – important to transform as robust methods assume data are normal once outliers are removed.

Non-log scale

H1N1 Zanamivir 2006/7

0.00

0.50

1.00

1.50

2.00

2.50

3.00

0 20 40 60 80 100

Virus

IC5

0

1.6

Log-scale (y-axis)H1N1 Zanamivir 2006/7

0.10

1.00

10.00

0 20 40 60 80 100

Virus

IC5

0

2.7

Alternative presentation – Box and Whisker Plot

Uses median + 1.5 * IQR (+2SD)Could be adjusted to match the scatter plots

3.0

H1N1 isolates 2004-2005 UK

1.4

2.4

2.1

1.8

2.3

1.95

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Oseltamivir Zanamivir

IC

50 (nM

)

Monitoring ICD50 (outliers) over time• First look at the data (scatter plots/box-whisker)

• Calculate statistics for different time periods– Median– %Outliers– Geometric means with 95% CI with outliers removed– Kruskal Wallis test for medians– Calculate Geometeric Means with 95% CI and t-test/

anova (outliers removed) – regression for time trend (outliers removed).

Monitoring ICD50 (outliers) over time

• Perform statistical tests– Kruskal Wallis test for medians

– Chi-squared or Fishers exact test for % outliers

– Anova or t-tests for comparing means (oultiers removed). Alternatively look for non-overlapping 95% CI which is a conservative method (approx p<0.007)

Graphical Presentation – Box and Whisker

H1N1 isolates 2004-2007 EU+ UK: Oseltamivir

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

H1N1 EU+UK 04-05 H1N1 EU+UK 05-06 H1N1 EU+UK 06-07

IC50 (

nM

)

Scatter Plot

H1N1 Zanamivir 2005-2007

0.10

1.00

10.00

IC50

2005/06 2006/07

3SD

1.65 SD

Median

Scatter plotH3N2 Zanamivir 2005-2007

0.10

1.00

10.00

100.00

IC50

2005/06 2006/07

3SD

1.65 SD

Median

Statistics H3N2 Zanamivir NIMR

Year Samples >3SD (%) median Geomean (95% CI)2005/06 88 1 (1.1%) 0.67 0.69 (0.62-0.76)2006/07 173 4 (2.3%) 0.59 0.58 (0.56-0.61)

• P-value comparing %>3SD = 0.67• P-value comparing medians = 0.0002 – significant although

actual difference small.• 95% CI for Geometric means do not overlap• Note of caution: Changes may occur within a year so this

comparison maybe too simplistic. For example there appears to be some evidence of a change within 2006/07

Summary

• Good use of statistical methods can help interpret the IC50 results and ensure assay results are reliable.

• Many appropriate methods already in place but more could be incorporated