Page 1
©2006 Millennium Pharmaceuticals, Inc. 1
Techniques in High-Content Screening and Assay Development
Workshop
Developing a Robust Automated Image-Based Assay
Doug Bowman,Sr. Mgr Imaging Sciences
Millennium Pharmaceuticals
Drug Discovery TechnologyAugust 7, 2006
Page 2
©2006 Millennium Pharmaceuticals, Inc. 2
The Big PictureThe Far Side by Gary Larson
• Cell-based assay• Immunofluorescence• Technologies / Instrumentation• Screening Assay• People
Page 3
©2006 Millennium Pharmaceuticals, Inc. 3
• IF Staining– Fix cells to substrate– Permeabilize membranes– Block – Add primary– Add secondary
• Image Acquisition• Image Analysis
HCS Assays using Immunofluorescence
Page 4
©2006 Millennium Pharmaceuticals, Inc. 4
• Provide physiological context for assaying drug activity via the quantitative analysis of cellular and sub-cellular events.
• Analysis also allows for the classification of cellular phenotypes based on morphological parameters.
“HIGH CONTENT”
Cellular “High Content” Imaging
Translocation
Protein Phosphorylation
NeuriteOutgrowth
Mitotic Index
Spindle Assembly / Morphology
Page 5
©2006 Millennium Pharmaceuticals, Inc. 5
Tissue…
• High content also!– Many of same assays
• Similar issues• High resolution
scanning
100um 20x objective
Page 6
©2006 Millennium Pharmaceuticals, Inc. 6
Technologies
• Instrumentation– Liquid handling robotics– Automated microscope-based
systems capable of acquiring high resolution images
• Software algorithms
OperaAutomatedMicroscope
Discovery-1
AcapellaProGuruMetaXpress
InCell1000
ArrayScan
Page 7
©2006 Millennium Pharmaceuticals, Inc. 7
Image-based assays
= 2λ / image * 5 sites / sample * 3 mice / timepoint= 30 images / timepoint= 400+MB / 5 timepoints
• Large volume of data
= 2λ / image * 9 sites / well * 96 wells / plate= 1728 images / plate = 1GB+ / plate3 plates / hr (Opera) 1 plate / hr (Discovery-1)
= 3λ / image * 15 sites / membrane= 30 images / sample= 120MB / sample
Page 8
©2006 Millennium Pharmaceuticals, Inc. 8
Working Group
Cell Biology Automation
Imaging
Assay Design
SoftwareDevelopment
ComputationalScience
Page 9
©2006 Millennium Pharmaceuticals, Inc. 9
Developing a Robust Automated Image-Based Assay:
Guidelines
• Use automation where possible• Visualize images• Develop robust process controls• Use data-driven approach
Page 10
©2006 Millennium Pharmaceuticals, Inc. 10
A data-driven approach to assay design
• Dissect components of the assay• Using control experiments
– Identify drivers of variability• Minimize them experimentally or account for them statistically
– Optimize parameter selection (robust)• Assess performance of assays in
project-relevant context
Page 11
©2006 Millennium Pharmaceuticals, Inc. 11
HCS Assay Components
• Dissect components• Using control experiments
– Identify drivers of variability– Minimize them experimentally
or account for them statistically
Wet LabProtocols
ImageAcquisition
Image Analysis
AssayAnalysis
Page 12
©2006 Millennium Pharmaceuticals, Inc. 12
HCS Assay Components
• Dissect components• Using control experiments
– Identify drivers of variability– Minimize them experimentally
or account for them statistically
Wet LabProtocols
ImageAcquisition
Image Analysis
AssayAnalysis
Page 13
©2006 Millennium Pharmaceuticals, Inc. 13
Wet Lab Protocols
Reagents
Automation: liquid handling IF Staining Protocol
Automation: cell plating
Reproducibility
Day to day variability
Page 14
©2006 Millennium Pharmaceuticals, Inc. 14
Older Multidrop tube cassettes found inaccurate – relative inaccuracy ~25% and CVs ~2%
New cassettes purchased and validated –relative inaccuracy ~2% and CVs ~0.7%
100ul manual (12 channel Multipipette) 4.43%100ul Multidrop 1.67%50ul Platemate 0.75%
50ul Tecan 0.80%10ul Precision 2000 1.34%
CVs
• Accuracy and Reproducibility– Compound addition / dilution– Cell Plating– Antibody addition / protocols
Wet Lab Protocols:
Testing liquid handling using Artel system
Artel MVS
Page 15
©2006 Millennium Pharmaceuticals, Inc. 15
Facilitate a systematic comparison of a large number of assay parameters to maximize data quality while minimizing the time required to find optimal parameters.
- fix- perm- block- primary ab dilution - secondary ab dilution- buffers- wash buffers- number of washes- time/temp of incubation….etc
Wet Lab Protocols:
IF Staining Protocol
Page 16
©2006 Millennium Pharmaceuticals, Inc. 16
Antibody Optimization
8 row trough
John Donovan
Matrix of 6 fixatives - 4 blocks - 4 abdilutions to assay
96 individual conditions
Page 17
©2006 Millennium Pharmaceuticals, Inc. 17
PBS TBS MTSB
1:500
1:750
1:1000
0
4 Abdilutions
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
PFA PBS PFA TBS PFA MTSB
Roche
BSA
PBS
Mixed Serum
MeOH Perm buf 1 MeOH Perm buf 2 MeOH Perm buf 3
4 Abdilutions
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
1
2
PFA PBS PFA TBS PFA MTSB
Roche
BSA
PBS
Mixed Serum
MeOH Perm buf 1 MeOH Perm buf 2 MeOH Perm buf 3
Wet Lab Protocols:
Antibody Optimization
Page 18
©2006 Millennium Pharmaceuticals, Inc. 18
DNA / Tubulin
Before After
• Comparison of > 400 parameters (~2 weeks)• Enhances analysis of spindle morphology• Save $ Antibody concentration
Scatter P lot
Calculated Column (2)01 02 03 04 05 06 07 08 09 10 11 12
H
G
F
E
D
C
B
A
S/N
Wet Lab Protocols:
Antibody Optimization
Page 19
©2006 Millennium Pharmaceuticals, Inc. 19
HCS Assay Components
• Dissect components• Using control experiments
– Identify drivers of variability– Minimize them experimentally
or account for them statistically
Wet LabProtocols
ImageAcquisition
Image Analysis
AssayAnalysis
Page 20
©2006 Millennium Pharmaceuticals, Inc. 20
Image Acquisition
Auto-Focus
Depth of Field Image Corrections
Instrument Choice
Reproducibility
Cell Plating: # sites
Illumination System
Page 21
©2006 Millennium Pharmaceuticals, Inc. 21
Image Acquisition
Auto-Focus
Depth of Field Image Corrections
Instrument Choice
Reproducibility
Cell Plating: # sites
Illumination System
Page 22
©2006 Millennium Pharmaceuticals, Inc. 22
Image Acquisition:
AutoFocus
• Image-based and Laser-based%Nuclear count in HT29 cells @ 24 hours
Millennium Pharmaceuticals Inc.
ML00618017-001-A
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic0
10
20
30
40
50
60
70
80
90
100
95%CI = 0.1498 µMEC30 = 1.51734 µM
Page 23
©2006 Millennium Pharmaceuticals, Inc. 23
60 wells, 9 sites / well
Millennium Pharmaceuticals Inc.
AVG %Ratio Nucl/Cyto in U2OS cells
[CMPD] (µM)0.01 0.1 1 10
AV
G %
Int r
atio
0
20
40
60
80
100
12095%CI = 0.2107 µM
IC50 = 0.57859 µM
Image Acquisition:
AutoFocus
Page 24
©2006 Millennium Pharmaceuticals, Inc. 24
• Minimize experimentally– Instrument parameters– Plate selection
• Identify, Remove: Auto-focus Classifier– Classify several hundred images by hand– Use image processing measures to extract
quantities that vary based on edges and intensity
– Train a classifier to call out-of-focus wells
Image Acquisition:
AutoFocus
Page 25
©2006 Millennium Pharmaceuticals, Inc. 25
-40 um -20 um 0 um 20 um 40 um
60
70
80
90
100
110
-50 -40 -30 -20 -10 0 10 20 30 40 50
Z Position (um)
Tota
l Nuc
lei
Original
Segmented
Image Acquisition:
AutoFocus
Page 26
©2006 Millennium Pharmaceuticals, Inc. 26
• BayesNet classifier learns differences between in-focus and out-of-focus images
• Algorithm maximizes separation of classes on training set by:
– automatically assigning weights for each predictor
– automatically selecting threshold
Δ IntensitySharpness 2 Δ AreaSharpness 1
Total score
0.080.21 0.18
0.55
yes>1.4?in focus
noout-of-focus
Threshold
Belief parameters
Image Acquisition:
AutoFocus
Page 27
©2006 Millennium Pharmaceuticals, Inc. 27
• Results of BayesNet assessed using cross-validation:
– Algorithm is trained on 9/10ths of the data and tested on remaining 1/10th
– Process is repeated 10 times – Error rates are averaged
• 93% to 98% of the wells were called correctly (for 3 different cell types).
• Nearly all classification errors occurred in 'marginal' category
• Zero-R (baseline classification error)= 32%
•
Image Acquisition:
AutoFocus
Page 28
©2006 Millennium Pharmaceuticals, Inc. 28
Filtering out-of-focus images via machine learning
Millennium Pharmaceuticals Inc.[CMPD] (µM)
0.01 0.1 1 10
AV
G %
Int r
atio
0
10
20
30
40
50
60
70
80
90
100
110
120
130
95%CI = 0.2401 µM
IC50 = 0.58004 µM
Millennium Pharmaceuticals Inc.[CMPD] (µM)
0.01 0.1 1 10
AV
G %
Int r
atio
0
10
20
30
40
50
60
70
80
90
100
110
120
130
95%CI = 0.0711 µM
IC50 = 0.6034 µM
Original Data Filtered Data
Image Acquisition:
AutoFocus
Page 29
©2006 Millennium Pharmaceuticals, Inc. 29
Image Acquisition
Auto-Focus
Depth of Field Image Corrections
Instrument Choice
Reproducibility
Cell Plating: # sites
Illumination System
Page 30
©2006 Millennium Pharmaceuticals, Inc. 30
• What is it?– Fluorescence illumination system
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
1 33 65 97 129 161 193 225 257 289 321 353 385 417 449 481 513 545 577 609 641 673
Uniform sample
Image Acquisition:
Illumination system variation
Page 31
©2006 Millennium Pharmaceuticals, Inc. 31
• Is it a problem? ↓ Cell counting assays↑↑ Cell intensity assays↑↑ Phenotyping
Flat-Field Correction
0.94
0.96
0.98
1
1.02
1.04
1.06
1.08
1.1
1.12
Nuclei Count Nuclei Intensity Tubulin Intensity
%Non FF CorrectedFF Corrected
Image Acquisition:
Illumination system variation
Page 32
©2006 Millennium Pharmaceuticals, Inc. 32
Image Acquisition
Auto-Focus
Depth of Field Image Corrections
Instrument Choice
Reproducibility
Cell Plating: # sites
Illumination System
Page 33
©2006 Millennium Pharmaceuticals, Inc. 33
Image Acquisition:
Depth of Field
20x
Page 34
©2006 Millennium Pharmaceuticals, Inc. 34
- 4.0 um- 2.0 um0.0 um2.0 um4.0 um
Image Acquisition:
Depth of Field
Page 35
©2006 Millennium Pharmaceuticals, Inc. 35
Z = -4.0um Z = -2.0um Z = 0.0um Z = 2.0um Z = 4.0um
“Best Focus”Single Plane
Image Acquisition:
Depth of Field
Page 36
©2006 Millennium Pharmaceuticals, Inc. 36
Outof
focus
Multiple optical sections @ 2.0um intervals
In focus
“Best Focus” Image
Tissue Imaging• Challenge: depth of focus of the objective is less than
the flatness of the specimen. This results in portions of a single image plane being out of focus.
Page 37
©2006 Millennium Pharmaceuticals, Inc. 37
HCS Assay Components
• Dissect components• Using control experiments
– Identify drivers of variability– Minimize them experimentally
or account for them statistically
Wet LabProtocols
ImageAcquisition
Image Analysis
AssayAnalysis
Page 38
©2006 Millennium Pharmaceuticals, Inc. 38
Image Analysis
Cell Plating: density
Algorithm parameters Signal / Noise
Threshold algorithm
Reproducibility
Magnification
Page 39
©2006 Millennium Pharmaceuticals, Inc. 39
Image Analysis:
Cell Plating Density• Assess
– Optimal cell plating density to minimize segmentation errors
• DOE:– Serial dilution– DAPI– 6 replicates
XXXXXXXXXXXXH
XXG
XXF
XXE
XXD
XXC
XXB
XXXXXXXXXXXXA
121110987654321
Cell plating density
Cell plating density
Cell plating density
Cell plating density
Cell plating density
Cell plating density
Page 40
©2006 Millennium Pharmaceuticals, Inc. 40
Image Analysis:
Cell Plating DensityCell Plating vs. Count
0
500
1000
1500
2000
2500
3000
Cell Plating Density
Nucl
ei C
ount
Page 41
©2006 Millennium Pharmaceuticals, Inc. 41
Image Analysis:
Threshold algorithm
0
200
400
600
800
1000
1200
1400
1 3 5 7 9
Drug Concentration
Nuc
lei C
ount
Object CountStd Area CountAdaptive Count
Page 42
©2006 Millennium Pharmaceuticals, Inc. 42
Image Analysis:
Signal / Noise• Assess
– Optimal exposure times to minimize acquisition time and maximize Signal/Noise
• Image-based autofocus• Intensity measurements• Segmentation
– DOE:– Uniformity Plate– DAPI– Multiple exposures
XXXXXXXXXXXXH
XXG
XXF
XXE
XXD
XXC
XXB
XXXXXXXXXXXXA
121110987654321
Page 43
©2006 Millennium Pharmaceuticals, Inc. 43
• Single Uniformity Plate, 9 sites/well
Decreasing exposure time
Image Analysis:
Signal / Noise
Page 44
©2006 Millennium Pharmaceuticals, Inc. 44
Adaptive Threshold
0
20
40
60
80
100120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12
Row
Nuc
lei C
ount
3.912.772.111.581.33
Fixed Threshold - Auto
020
4060
80100120
140160
180200
1 2 3 4 5 6 7 8 9 10 11 12
Row
Nuc
lei C
ount
Fixed Threshold - User
0
20
40
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12
Row
Nuc
lei C
ount
• Effect on thresholding
Image Analysis:
Signal / Noise
Page 45
©2006 Millennium Pharmaceuticals, Inc. 45
Count Nuclei
0
0.2
0.4
0.6
0.8
1
1.2
3.91 2.77 2.11 1.58 1.33
eSNR
% o
f 160
*: p<10-100, Welch’s two-tailed t-testError bars are standard deviations
*
*
Image Analysis:
Signal / Noise
Page 46
©2006 Millennium Pharmaceuticals, Inc. 46
• What objective to use?– Segmentation accuracy– Dose response curve re-sampling
Image Analysis:
Magnification
Page 47
©2006 Millennium Pharmaceuticals, Inc. 47
10x
20x
FKHR-GFPDNA
Image Analysis:
Magnification
Page 48
©2006 Millennium Pharmaceuticals, Inc. 48
10x
20x
FKHR-GFPDNA
Image Analysis:
Magnification
Page 49
©2006 Millennium Pharmaceuticals, Inc. 49
Millennium Pharmaceuticals Inc.[CMPD] (µM)
0.01 0.1 1 10
AV
G %
Int r
atio
0
10
20
30
40
50
60
70
80
90
100
110
120
130
95%CI = 0.0744 µM
IC50 = 0.65515 µM
Millennium Pharmaceuticals Inc.[CMPD] (µM)
0.01 0.1 1 10
AV
G %
Int r
atio
0
10
20
30
40
50
60
70
80
90
100
110
120
130
95%CI = 0.0372 µM
IC50 = 0.64973 µM
10x 20x
Image Analysis:
Magnification
Page 50
©2006 Millennium Pharmaceuticals, Inc. 50
• Assess– Effect of variability of user-defined parameters– Different software algorithms (vendors)
• DOE:– Dose Response– Multiple algorithmparameters
XXXXXXXXXXXXH
XCXG
XCXF
XCXE
XCXD
XCXC
XCXB
XXXXXXXXXXXXA
121110987654321
dose response
dose response
dose response
dose response
dose response
dose response
Image Analysis:
Algorithm parameters
Page 51
©2006 Millennium Pharmaceuticals, Inc. 51
• User variability?
Image Analysis:
Algorithm parameters
Page 52
©2006 Millennium Pharmaceuticals, Inc. 52
• Single Plate, 9 sites/well, dose-response• 30 variations of DNA, pH3 threshold
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0499µMIC30= 0.65911µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0499µMIC30= 0.65911µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0594µMIC30= 0.65607µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
TEST
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0462µMIC30= 0.60712µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0652µMIC30= 0.65776µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.05µMIC30= 0.61013µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.1432µMIC30= 0.66916µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0953µMIC30= 0.66289µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.198µMIC30= 0.66241µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0901µMIC30= 0.66033µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0605µMIC30= 0.65709µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0439µMIC30= 0.65786µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0439µMIC30= 0.65786µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0634µMIC30= 0.65637µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
TEST
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0442µMIC30= 0.60945µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0664µMIC30= 0.65259µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0448µMIC30= 0.60799µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.1733µMIC30= 0.66231µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0858µMIC30= 0.65996µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.1274µMIC30= 0.66117µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0707µMIC30= 0.65613µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.046µMIC30= 0.65204µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0476µMIC30= 0.65429µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0463µMIC30= 0.6547µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0463µMIC30= 0.6547µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0627µMIC30= 0.6526µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
TEST
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0404µMIC30= 0.6082µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0999µMIC30= 0.66593µM
%Nuclear count in HT29 cells @ 24 hour
Millennium Pharmaceuticals Inc.
[CMPD] (µM)0.1 1 10
AV
G %
Mito
tic
0
10
20
30
40
50
60
70
80
90
100
95%CI= 0.0685µMIC30= 0.65928µM
IC30 (µM)
0.1000.2000.3000.4000.5000.6000.7000.8000.9001.000
Algorithm Parameters (DNA, pH3 adaptive threshold)
IC30
(µM
)
DNApH3
Image Analysis:
Algorithm parameters
Page 53
©2006 Millennium Pharmaceuticals, Inc. 53
HCS Assay Components
• Dissect components• Using control experiments
– Identify drivers of variability– Minimize them experimentally
or account for them statistically
Wet LabProtocols
ImageAcquisition
Image Analysis
AssayAnalysis
Page 54
©2006 Millennium Pharmaceuticals, Inc. 54
Assay Analysis
# of replicates
Transfer metrics Variance Component Analysis
(Plate-Plate / Day-Day)
Instrument Choice
Reproducibility
96 vs 60 wells
# images / well
Page 55
©2006 Millennium Pharmaceuticals, Inc. 55
Assay Analysis
# of replicates
Transfer metrics Variance Component Analysis
(Plate-Plate / Day-Day)
Instrument Choice
Reproducibility
96 vs 60 wells
# images / well
Page 56
©2006 Millennium Pharmaceuticals, Inc. 56
Assay Analysis:
Variance Component Analysis
• Assess variability– Row / column– Plate-plate– Day-day
• DOE:– Single drug concentration (Uniformity plates)– 4 plates x 3 days
Day 1 Day 2 Day 3
Page 57
©2006 Millennium Pharmaceuticals, Inc. 57
96 wells 60 wells
• Inner 60 wells had median CV which was 35% lower than that for all 96 wells
Assay Analysis:
Uniformity Data
Page 58
©2006 Millennium Pharmaceuticals, Inc. 58
Single Plate
12 Plates •Variability among days was largest component followed by plate•Replicate dose response curves should be run on different plates on different days to account for largest amount of variability.
Total Mean: 27.7Total Variance: 32.0 (std dev: 5.7)Coefficient of Variation: 20.4%
Assay Analysis:
Uniformity DataVariance Component Analysis
Page 59
©2006 Millennium Pharmaceuticals, Inc. 59
Assay Analysis:
Re-sampling
• Determine– How many sites / well?– How many replicate wells?– Is the assay robust?– Does assay perform well across facilities?
• Sample– Uniformity Plates– Control Compound– Multiple Compounds
Page 60
©2006 Millennium Pharmaceuticals, Inc. 60
Bootstrapping
Population
• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample
Page 61
©2006 Millennium Pharmaceuticals, Inc. 61
Bootstrapping
Population Sample
• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample
Page 62
©2006 Millennium Pharmaceuticals, Inc. 62
Bootstrapping
Population Sample
• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample
Page 63
©2006 Millennium Pharmaceuticals, Inc. 63
Bootstrapping
Population Sample
• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample
Page 64
©2006 Millennium Pharmaceuticals, Inc. 64
Bootstrapping
Population Sample
• Method for estimating the sampling distribution of an estimator by re-sampling with replacement from the original sample
Page 65
©2006 Millennium Pharmaceuticals, Inc. 65
Assay Analysis:
Control Compound• Assess
– Effect of number of replicate curves upon EC30 estimates
– Effect of number of images per well upon EC30 estimates
• DOE:– 6 dose response
curves– 5 plates
XXXXXXXXXXXXH
XCXG
XCXF
XCXE
XCXD
XCXC
XCXB
XXXXXXXXXXXXA
121110987654321
dose response
dose response
dose response
dose response
dose response
dose response
Page 66
©2006 Millennium Pharmaceuticals, Inc. 66
Assay Analysis:
Control Compound Data
Page 67
©2006 Millennium Pharmaceuticals, Inc. 67
Re-sampling Results
Distribution of EC30 values
Page 68
©2006 Millennium Pharmaceuticals, Inc. 68
Error bars are 95% CI All CVs are less than 12%
-Results incorporate differences among plates and there were fourobservations with high values.
Re-sampling Results:
Determine # wells
Page 69
©2006 Millennium Pharmaceuticals, Inc. 69
Going from 6 images to 9 has almost as much impact on standard error as going from 9 to 16.
10uM DMSO
Re-sampling Results:
Determine # sites/well
Page 70
©2006 Millennium Pharmaceuticals, Inc. 70
Assay Analysis
# of replicates
Transfer metrics Variance Component Analysis
(Plate-Plate / Day-Day)
Instrument Choice
Reproducibility
96 vs 60 wells
# images / well
Page 71
©2006 Millennium Pharmaceuticals, Inc. 71
Assay Analysis:
Transfer
• Assess– Compare re-sampled EC30 estimates to estimates
generated in Oncology Discovery• DOE
– 4 compounds– 3 replicates / compound– Each compound was run on 4 plates = 12 curves– All plates run in two facilities
Page 72
©2006 Millennium Pharmaceuticals, Inc. 72
Four compound re-sampling results
•A and B were not significantly different, C and D were approximately 60% higher
• Three wells from each concentration were sampled.• Fit dose response curve and calculated EC30.• Total of 250 curves• Examine distribution of EC30 values
Compound
Log(
EC30
)
Assay Analysis:
Transfer
Page 73
©2006 Millennium Pharmaceuticals, Inc. 73
• Uniformity Plates
• Single Compound
Assay Analysis:
Instrument selection
Discovery-1 Opera
* determination based on signal/noise, speed, robustness, availability
• Signal/Noise, acquisition time, CVs
• Z’, EC50 CVs, confidence interval
Page 74
©2006 Millennium Pharmaceuticals, Inc. 74
Overall Assay Evaluation
• ~2 months• Confidence
– Immunofluorescence staining protocol
– Automation / liquid handling– Instrument / Instrument QC– Software Algorithm – Robust assay– Transfer from Discovery to
Platform Group
Platform validation
Variance ComponentsAnalysis
Bootstrapping on wellsand images
Bootstrapping withcontrol compound
Bootstrapping withreference panel
Identify sources of noise
Format assay
Determine robustness
Make site comparisons
Establish decision criteriaDecision criteria
simulation
Process controls
Platform validation
Variance ComponentsAnalysis
Bootstrapping on wellsand images
Bootstrapping withcontrol compound
Bootstrapping withreference panel
Identify sources of noise
Format assay
Determine robustness
Make site comparisons
Establish decision criteriaDecision criteria
simulation
Process controls
Page 75
©2006 Millennium Pharmaceuticals, Inc. 75
Summary
• Look at the images
• Run uniformity plates to identify and minimize noise (biology, instrument, and algorithm)
• Use analysis modules throughout process
Page 76
©2006 Millennium Pharmaceuticals, Inc. 76
Process Deliverables
• Development Guidelines– QC metrics: instruments, algorithm– IF staining protocol optimization– Image analysis optimization
• Production Guidelines – Statistical analysis for transfer– Ongoing QC metrics
Page 77
©2006 Millennium Pharmaceuticals, Inc. 77
Acknowledgements• Molecular & Cellular Oncology
– Ben Amidon– Jeff Ecsedy– Natalie Roy D’Amore– Liz Carideo– Kara Hoar – Claudia Rabino– Michelle Tighe-Nestor, – Denise Driscoll– Deborah Wysong
• Cancer Pharmacology– Arijit Chakravarty– Katherine Galvin
• Lead Discovery– John Ringeling– John Donovan
• Computational Sciences– Sudeshna Das– Mike Pickard
Page 78
©2006 Millennium Pharmaceuticals, Inc. 78
Breakthrough science. Breakthrough medicine.SM