1 Key Objectives Parallel Programming Model and Tools desesperatly needed for the masses (New Scientist, New SME) for new architectures (Multi-cores) As Effective as possible: Efficient However Programmer/User Productivity is first Key For both Multi-cores and Distributed Actually the way around Some Handling of ``Large-scale’’ (Grid, Clouds)
61
Embed
1 Key Objectives Parallel Programming Model and Tools desesperatly needed for the masses (New Scientist, New SME) for new architectures (Multi-cores)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Key Objectives Parallel Programming Model and Tools
desesperatly needed for the masses (New Scientist, New SME) for new architectures (Multi-cores)
As Effective as possible: Efficient However Programmer/User Productivity is first Key
For both Multi-cores and Distributed Actually the way around
Some Handling of ``Large-scale’’ (Grid, Clouds)
2
Intech, Jeudi 2 juillet 2009
3
D. Caromel, et al.
Overview of Cloud, Parallel Computingand ProActive PACA Grid
For: Science Labs and Local Industries (Large and SME)
47
Intech, Jeudi 2 juillet 2009
48
49
ProActive PACA Grid in Context
50
4. Use Case: IPMC
52
Use case: SOLiD and ProActive
SOLiD from Applied Biosystems (USA)
As part of a project with the IPMC research institute, the SOLiD Corona Lite software has been upgraded by integrating ProActive to enable the distribution of parallel tasks on lab desktops in order to accelerate the processing
At the moment, only the first pipeline, Matching, has been upgraded by distributing the Mapreads function
Constraints Requirements set by IPMC: keep the current
software architecture ProActive has been integrated on top of PBS
Matching
Pairing
SNP/Consensus calling
ProActive
PBS
Resource Manager
53
Resources set up
Environment
16 nodes
Additional external nodes can be easily and dynamically added!
54
Mapreads optimization
The Reads are split into smaller files
Each Reads subset is compared to one chromosome
The resulting files are merged
55
Optimized Mapreads Performances
The distributed version with ProActive of Mapreads has been tested on the INRIA cluster with two settings: the Reads file is split in either 30 or 10 slices
Use case: matching 31 millions sequences with the human genome (M=2, L=25)
Reference point with 16 cores(same as in SOLiD machine)
4 Time faster from 20 to 100Speed Up of
80 / Th. Sequential50 Hours 35 Minutes
56
Key benefits of this solution
Higher throughput Reduced execution time
Scalable Depending on the input and reference data size, the user can chose to increase or reduce
the number of extra resources used Solution is ready for next generation reads file
Flexible The run can be paused or resumed by the user when needed Priorities between jobs can be easily set by the users Easy nodes acquisition and hot plugging
Simplified maintenance ProActive directly supports common schedulers like PBS, LSF, SGE, and W HPCS 08:
time consuming adaptations of Corona Lite software are no longer needed
Reduced costs for Applied Biosystems customers Optimizing available hardware resources Free use of ProActive Parallel Suite® Easy to install and use: save time
Supported by experts in parallel computing
Accelerate & Scale up withProActive Parallel Suite®
02/07/2009
Presentation overview
Item1 Item 2
Item3
Thank you!
60
Co-developing, Support for ProActive Parallel Suite Worldwide Customers: Fr, UK, Boston USA