CLCAR, Panamá 2012 A Framework for High Performance Image Analysis Over Cloud Resources 1 A Framework for High Performance Image Analysis Pipelines over Cloud Resources Raúl Ramos-Pollán, Ángel Cruz-Roa, Fabio González-Osorio Bioingenium Research Group Universidad Nacional de Colombia
20
Embed
A Framework for High Performance Image Analysis Pipelines ... · CLCAR, Panamá 2012 A Framework forHigh Performance ImageAnalysisOverCloud Resources 1 A Framework for High Performance
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
1
A Framework for High Performance ImageAnalysis Pipelines over Cloud Resources
Raúl Ramos-Pollán, Ángel Cruz-Roa, Fabio González-Osorio Bioingenium Research Group
Universidad Nacional de Colombia
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
2
Motivation
• Bioingenium research group
www.bioingenium.unal.edu.co
• Large scale image processing pipelines
• Initially focused on medical imaging
• Support for machine learning processes
• Seldom availability of computing resources
• Limitation on collections size, applicability,algorithms design
1 Motivation
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
3
Strategy
• Build on experiences with Hadoop / Grid, etc.
• Profit from whatever resources available
• Decouple algorithm design from deployment
• Adopt Big Data principles and technologies
• Streamline software development process
• Unify coherent algorithms repository
1 Motivation
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
4
Image Processing Pipelines
2 Image Processing Framework
PATCH EXTRACTION
INPUTIMAGES
FEATURES EXTRACTION
FEATURESCLUSTERING
LATENTSEMANTICS
AUTOMATIC ANNOTATION
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
5
Image Processing Framework
2 Image Processing Framework
NOSQL DATABASE
CLIENT at EXPERIMENTER’S
DESKTOP
SCHEDULE GENERATOR
EXPERIMENTER
SCHEDULE
PIPELINE DEFINITION
STAGE 1 STAGE 2
STAGE 3STAGE 4
INPUTOUTPUT
DATA
WORKERS
(AMAZON, DESKTOPs)
WORKERADDED LATER
WORKER
WORKER
CLCAR, Panamá 2012
A Framework for High Performance Image Analysis Over Cloud Resources
6
Pipeline definition example
2 Image Processing Framework
######################################## FIRST STAGE: Patch Sampling#######################################stage.01.task: ROIsFeatureExtractionTaskstage.01.numberOfPartitions: 10stage.01.roiAlgorithm: RandomPatchExtractorstage.01.feAlgorithm: GrayHistogram
##################################### THIRD STAGE: Bag of Features Histograms####################################stage.03.task: BagOfFeaturesExtractionTask