New Capabilities in QosCosGrid Middleware for Advanced Job Management, Advance Reservation and Co-allocation of Computing Resources B. Bosak, P. Kopta, K. Kurowski, M. Mamonski, T. Piontek Poznan Supercomputing and Networking Center Cracow Grid Workshop 8-th November 2011
17
Embed
B. Bosak, P. Kopta , K. Kurowski , M. Mamonski , T. Piontek
New Capabilities in QosCosGrid Middleware for Advanced Job Management, Advance Reservation and Co- allocation of Computing Resources. B. Bosak, P. Kopta , K. Kurowski , M. Mamonski , T. Piontek Poznan Supercomputing and Networking Center. Cracow Grid Workshop 8-th November 2011. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
New Capabilities in QosCosGrid Middleware for Advanced Job Management, Advance Reservation and Co-allocation of Computing Resources
B. Bosak, P. Kopta, K. Kurowski, M. Mamonski, T. PiontekPoznan Supercomputing and Networking Center
Cracow Grid Workshop8-th November 2011
2
Plan of the Presentation
Introduction
Main features of QCG
Comparison with other
Grid systems
QCG Architecture and its main Components
Status on deployments
3
Introduction
Co-allocationAdvance Reservation
Large-scale parallel
applications
Cross-cluster MPI
and ProActive
Multiscale simulationsWorkflows
4
QCG for Parallel Applications
• Multicluster OpenMPI and ProActive• Hybrid applications, e.g. OpenMPI/OpenMP• Multiscale, cross-cluster applications based on
the MUSCLE framework.• Application consisting of groups of processes
with different resource requirements• Topology aware scheduling:
- by QCG basing on application requirements- by application basing on topology discovery
5
QCG for Workflow Applications
• Workflows based on direct acyclic graphs (DAG)
• Task may be triggered by statuses of proceeding tasks (e.g. some task may be started when the proceeding task is in a state “Running”)
• Multi dimensional parameter sweep experiments (as a part of workflow)
6
Advance Reservation and Co-allocation
• AR is a mechanism offering execution of applications in a specified timeslots
• Main use case in cross-cluster application execution, where co-allocation of resources is required
• QCG creates co-allocation based on parameters specified by users: it may use not only resource requirements, but also requested start time, end time or duration
7
QosCosGrid vs. Popular Grid Middleware
Middleware Single jobs Workflows MPI Jobs Cross cluster MPI jobs
Interactive Jobs
Parametric Jobs
gLite Yes Yes Yes No Yes Yes
UNICORE Yes Yes Yes No No Yes
QCG Yes Yes Yes Yes No Yes
8
QCG Architecture
9
QCG-Broker
• Grid domain meta-scheduling framework• Deals with load-balancing and scheduling of
cross-cluster jobs• Provides consisted WebService interface to
the Grid; JobProfile XML-based language is used as a job description format
• NEL, the application from Quantum Chemistry written by prof. Jacek Komasa was adapted to cross-cluster execution on top of QosCosGrid. Tests were performed on the PL-Grid infrastructure.
• Ongoing production deployments in Europe on EGI and PRACE resources (e.g. LRZ, UCL, SARA) – MAPPER project.
16
Summary
• QosCosGrid is an alternative grid middleware
• Great support for cross-cluster application execution (MPI, ProActive, MUSCLE)
• Ready production deployments in PL-Grid infrastructure.