Top Banner
Screen Mining with KNIME A user-friendly framework for high throughput / content data analysis Mar$n Stöter HT - Technology Development Studio (TDS), the HC-Screening Unit at the MPI-CBG [email protected] KNIME workshop February 27 th 2016, Berlin
36

Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Oct 19, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Screen Mining with KNIME

A user-friendly framework for high throughput / content data analysis

Mar$nStöterHT-TechnologyDevelopmentStudio(TDS),[email protected]

KNIMEworkshopFebruary27th2016,Berlin

Page 2: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Outline

Martin Stöter, MPI-CBG, Dresden, Germany 2

-  IntroducAonintoHigh-ContentScreening(HCS)dataandtheHCSToolsnodes

-  Hands-onsessionHCSTools-  IntroducAonintoScripAngIntegraAonnodes-  Hands-onsessionScripAonIntegraAon

Page 3: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Technology Development Studio (TDS)

Martin Stöter, MPI-CBG, Dresden, Germany 3

MPI-CBG,Dresden,Germany

Screeningfacilityforacademiclaboratories

ProvidefullserviceforautomaAonandcell-basedscreens,RNAiand

chemicalscreens

Equipment:liquidhandlingrobots,dropdispensers,platewashers,platereaders,

HighContentScreeningplaTorms

Page 4: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Data Analysis is a Bottleneck in HCS!

4

Dataanalyst

ComplexExperimentsLotsofdata(toomuchforExcel)Fancydataanalysis/miningManyscienAsts,butfewdataanalystsSomeAmesdifferentlanguagesDataanalysisisoYenaboZleneck!

Scien$sts

HCSTools+ +

+…

Page 5: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

High-Content Screening (HCS) data

Martin Stöter, MPI-CBG, Dresden, Germany 5

DatageneraAon-Cells(RNAi,compounds)-Microscopy->images-Imageanalysis-Cellfeatures/parameters->welldata

Tasks/problems-ReaddatafromvarioussourcesSQLdatabase,XML,Excel,various.csv…-ScreeningspecificstaAsAcs-ScreeningspecificuAliAes-Datamining,visualizaAon

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24AB DMSO DMSO DMSOC 0.001 DMSO DMSO 0.001D 10 DMSO DMSO 10E 10 DMSO DMSO 10F 3 DMSO DMSO 3G 3 DMSO DMSO 3H 1 DMSO DMSO 1I 1 DMSO DMSO 1J 0.3 DMSO DMSO 0.3K 0.3 DMSO DMSO 0.3L 0.1 DMSO DMSO 0.1M no AB no AB 0.1 DMSO DMSO 0.1N no AB no AB 0.1 DMSO DMSO 0.1O DMSO DMSOP

Page 6: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools for KNIME

DataImportImageAnalysisReaders(Opera,OpereQa,Mo$onTracking)PlateReaders(Envision,GeniusPro,MSDSectorImager)Other(ExampleData,GenericXML)

NormalizaAonPercent-of-control(POC),Normalizedpercentinhibi$on(NPI)Z-score,B-scoreVectorLengthNormaliza$on(clustering)Op$onal:robuststa$s$cs(Median+MAD)Selectwellstonormalize(controls,samples)

QualityControlZ-primefactor(Z‘),Mul$variateZ‘,SSMDCV(coefficientofvariance)Op$onal:robuststa$s$cs(Median+MAD)Selectwellstonormalize(controls,samples)

HCSTools

Page 7: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools for KNIME

UAliAesHandlebarcodes,wellsandrowleQersJoinLayoutfromExcel(wellannota$on,metadata)CreateWellPosi$on(NEW)

VisualizaAonPlateHeatmapViewerDoseResponse(dependentonR!)

AdvancedStaAsAcsBinningAnalysisDataManitupaAon/Pre-ProcessingSplit/CombineColums(byheader)NumberFormaQer(NEW)RangeFilter,SpliQerOutlierRemoval

HCSTools

Page 8: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Standardized Data Format

-EnforcestandardizaAonofdataformat

-Differentreadernodestoshapeacommondatastructure

-Lowertheknowledgeentrybarrierfornewusers

“barcode”,“plateRow”,“plateColumn”,param1,param2,…

->EasesuptheusageofotherHCSToolsnodes

Page 9: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Expand well

StandardizaAonofthewellcoordinates:-“plateRow”and“plateColumn”asintegervaluesresemblewellposi$onmatrix(insteadofwell)

-Somenodesselectthesecolumnsasdefault(JoinLayout,PlateHeatmapViewer)

-Compa$blewith96,384and1536wellformat

-PlateRowConverter(leQer↔integer)

-CreateWellPosi$on(sortablewellstring)

NEWNODE

Page 10: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Barcode Standard

RegularexpressionforinterpretaAonofbarcode:-Standardizedtablestructure->connec$ontoourTDScompounddatabase

-(?<libplatenumber>[0-9]{3})(?<projectcode>[A-z]{2})(?<date>[0-9]{6})(?<replicate>[A-z]{1})

-ConfigurableinPreferences->KNIME->HCATools

-Mul$plebarcodes/regularexpressionspossible

-Finalreleaserecently

Page 11: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Barcode Standard

RegularexpressionforinterpretaAonofbarcode:-Standardizedtablestructure->connec$ontoourTDScompounddatabase

-(?<libplatenumber>[0-9]{3})(?<projectcode>[A-z]{2})(?<date>[0-9]{6})(?<replicate>[A-z]{1})

-ConfigurableinPreferences->KNIME->HCATools

-Mul$plebarcodes/regularexpressionspossible

-Finalreleaserecently

Page 12: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Barcode Standard

RegularexpressionforinterpretaAonofbarcode:-Standardizedtablestructure->connec$ontoourTDScompounddatabase

-(?<libplatenumber>[0-9]{3})(?<projectcode>[A-z]{2})(?<date>[0-9]{6})(?<replicate>[A-z]{1})

-ConfigurableinPreferences->KNIME->HCATools

-Mul$plebarcodes/regularexpressionspossible

-Finalreleaserecently

Page 13: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Annotate Experiment

ExcelisthetoolforexperimentdocumentaAonandassaydevelopment

JoinLayoutnodeisExcelReaderfordefinedspreadsheet

PlateformatwithmulAplewellaZributes(1platelayout->1columninKNIME)

-TitleoflayoutstartsincellC5

-Twoemptyrowsbetweenthelayout

Page 14: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Normalization

Tocomparedatafromdifferentplates,daysorrunsdatamustbenormalizedperplate

SelectablereferencewellpopulaAonperplatePercent-of-control(POC),Normalizes-percent-of-

inhibiAon(NPI),Z-ScoreRobuststaAsAcs(median&madinsteadofmean&sd)

withstaAsAcstableassecondoutput

Page 15: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Normalization

Tocomparedatafromdifferentplates,daysorrunsdatamustbenormalizedperplate

SelectablereferencewellpopulaAonperplatePercent-of-control(POC),Normalizes-percent-of-

inhibiAon(NPI),Z-ScoreRobuststaAsAcs(median&madinsteadofmean&sd)

withstaAsAcstableassecondoutput

Page 16: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Normalization

Tocomparedatafromdifferentplates,daysorrunsdatamustbenormalizedperplate

SelectablereferencewellpopulaAonperplatePercent-of-control(POC),Normalizes-percent-of-

inhibiAon(NPI),Z-ScoreRobuststaAsAcs(median&madinsteadofmean&sd)

withstaAsAcstableassecondoutput

Page 17: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Quality Control (QC)

QualitycontrolstaAsAcmeasuretheassayperformanceSelectable(mulAple)referencewellpopulaAonperplateZ-Primefactor(Z’),mulAvariateZ’,strictlystandardized

meandifference(SSMD),coefficientofvariance(CV)RobuststaAsAcs(median&madinsteadofmean&sd)

Page 18: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Quality Control (QC)

QualitycontrolstaAsAcmeasuretheassayperformanceSelectable(mulAple)referencewellpopulaAonperplateZ-Primefactor(Z’),mulAvariateZ’,strictlystandardized

meandifference(SSMD),coefficientofvariance(CV)RobuststaAsAcs(median&madinsteadofmean&sd)

Page 19: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Quality Control (QC)

QualitycontrolstaAsAcmeasuretheassayperformanceSelectable(mulAple)referencewellpopulaAonperplateZ-Primefactor(Z’),mulAvariateZ’,strictlystandardized

meandifference(SSMD),coefficientofvariance(CV)RobuststaAsAcs(median&madinsteadofmean&sd)

Page 20: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Binning Analysis

BinninganalysisdescribeschangesindistribuAonsGreattoolformovingfromcelltowelldata(insteadof

justtakingmeanperwell)

"CellProfiler and KNIME: open source tools for high content screening.". Methods in molecular biology (Clifton, N.J.) 2013 986, S. 105-22

Page 21: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Binning Analysis

BinninganalysisdescribeschangesindistribuAonsGreattoolformovingfromcelltowelldata(insteadof

justtakingmeanperwell)

"CellProfiler and KNIME: open source tools for high content screening.". Methods in molecular biology (Clifton, N.J.) 2013 986, S. 105-22

Page 22: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Binning Analysis

BinninganalysisdescribeschangesindistribuAonsGreattoolformovingfromcelltowelldata(insteadof

justtakingmeanperwell)

"CellProfiler and KNIME: open source tools for high content screening.". Methods in molecular biology (Clifton, N.J.) 2013 986, S. 105-22

Page 23: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Viewer (discontinued)

Martin Stöter, MPI-CBG, Dresden, Germany 23

179platesx384wells=~70.000datapointsAmesxparameters

Page 24: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

Page 25: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

Page 26: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 27: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 28: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 29: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 30: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 31: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 32: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 33: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: Plate Heatmap Viewer

VisualizaAonofscreeningcampaignswithmetadataEasytofindvisuallypaZers,driYs,errors…Newfeatures:

-KNIMEColors-HiLitesupport-representaAonofimages-manydifferentconfiguraAons,e.g.colorscale…

-10x384wellplate-3replicates-~10,000datapoints-Rawdata-Metadatafrombarcode-Normalizeddata-Differentreadout-Metadatafromlayout-Browsingsingleplate-Viewingthewelldata-Displayofimages-…more

Page 34: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: what was / is cooking?

Newnodes-CreateWellPosiAon-NumberFormaZer

Enhancements-DoseResponse(R)

PlateViewerwasdisconAnuedBinningAnalysisworkinprogress

-BinningCalculate

-BinningApply

-BinningQC&ModelModifier

Transformsnumberstodefinedstring

-imageoutput(insteadofview)-moresta$s$csintableoutput(e.g.Hillcoefficent)-moreplotop$ons(SEM)-newmodelport

?

Page 35: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

HCS Tools: the demo

Ok…nowlet’sgototheworkflowandseethenodes…

Thedataset:CellProfilerImagedata(pre-cleanedupasa.tabledueto

technicalreasons)

-10x384wellplatesin3replicateswith3imagesperwell

Page 36: Screen Mining with KNIME - idisk-srv1.mpi-cbg.de€¦ · Outline Martin Stöter, MPI-CBG, Dresden, Germany 2 - IntroducAon into High-Content Screening (HCS) data and the HCS Tools

Acknowledgements

36

SoYwareDevelopmentAntjeJanoschTimNicolaisenMagdalenaRucinskFelixMeyerhofer(past)HolgerBrandl(past)

HCSTools

TDSteam(MPI-CBG)

KNIMEMichaelBertholdandtheKNIMEteam