Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis Srikumar Venugopal 1, Rajkumar.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Slide 1
Gridbus Resource Broker for Application Service Costs-based
Scheduling on Global Grids: A Case Study in Brain Activity Analysis
Srikumar Venugopal 1, Rajkumar Buyya 1, Susumu Date 2 1.Grid
computing & Distributed Systems (GRIDS) Lab. The University of
Melbourne Melbourne, Australia www.gridbus.org/www.gridbus.org/ 2.
Cybermedia Center, Osaka University GRIDS
Slide 2
What does a Resource Broker do? Gets user/application
requirements Discovers resources like computational nodes, data
sources, etc. Establishes costs, user credit, etc. Makes decisions
about the optimal schedule for jobs Dispatches jobs Resource Broker
Cataloguing Services Accounting Services Information Services
Application Grid Nodes
Slide 3
Architecture of Gridbus Scheduler Data Catalogue ASP Catalogue
Gridbus Scheduler Grid Info Service Grid Market Directory Grid Node
GridBank Agent+Job Grid Node (e.g., UofM) PE Grid Node (e.g., VPAC)
PE Grid Node (e.g., ANL) CPU or PE Access Service (Globus) Globus
GTS Cluster Scheduler Application Results Bill Visual Parametric
Tool
Slide 4
Gridbus Scheduler Interfaces with: Application Development -
Visual Parametric Tool Information Services - Grid Market Directory
(Cost), GRIS,etc. Accounting Services - Grid Trading Service,
GridBank Cataloguing Services - Application Catalog, Replica
Catalog Job Dispatcher Nimrod-G (for parametric jobs) Gridbus
Dispatcher (for data intensive, reservation, P-GRADE support, etc.)
work in progress Supports: User-specified QoS parameters such as
Deadline, budget, etc. Application Cost or Hardware Cost (CPU, etc)
Cost from Grid Market Directory or Flat File Cost, Time or
Cost-Time Optimization.
Slide 5
Application Service Costs? Present Approach to Processing Cost
- Timeshare or CPU cycles used Users more interested in the cost of
getting job done than amount of processing power consumed New
Approach to Cost - Application Service Costs charge for using the
application once. Different costs for different applications
depends on provider Broker finds Cost through Grid Market
Directory.
Slide 6
Scheduling Algorithms Gridbus Scheduler implements Cost
Optimization Minimize computational cost (within deadline) Time
Optimization Minimize execution time (within budget) Cost-time
Optimization Similar to cost-optimization Implemented for first
time.
Slide 7
Scheduling (contd..) Uses past performance to forecast each
machines capacity The rate of completion is averaged to compensate
for any spikes or troughs Cost Optimization Gives maximum jobs to
the cheapest machine Time Optimization Gives jobs to machines based
on consumption rate but limited by budget per job Cost-Time
Optimization Distributes jobs among the machines of consumption
sorted by their consumption rate
Slide 8
Cost Optimization: No. of Jobs Done vs time
Slide 9
Cost-Time Optimization: No. of Jobs Done vs Time
Slide 10
Time Optimization: No. of Jobs Done vs Time
Slide 11
Comparison of Scheduling Algorithms All experiments were
started with No of Jobs = 200 Deadline = 2hrs Budget = 600 Grid $
Start TimeCompletion TimeBudget Consumed (Grid $) Cost10:00
a.m.11:27 a.m.188 Cost-Time11:40 a.m.12:08 p.m.277 Time12:30
p.m.12:59 p.m.274
Slide 12
Case Study: Brain Activity Analysis In Collaboration with Osaka
University, Japan Computationally and data intensive
Slide 13
MEG Data/Brain Activity Analysis MEG (Magnetoencephalography)
Achieve both non-invasiveness and high degree of measurement
accuracy cf. EEG (Electroencephalography), ECoG
(Electrocorticography) Measure functional data on multiple points
around the head Promising among medical doctors and brain
scientists. A B A: B: http://www.ctf.com
Slide 14
MEG data analysis Osaka Univ. Hospital Osaka Univ. DV transfer
Life-electronics laboratory, AIST Data Analysis Provision of MEG
Provision of expertise in the analysis of brain function Cybermedia
Center Data Generation Analysis Results
Slide 15
MEG data analysis Osaka Univ. Hospital Osaka Univ. DV transfer
Life-electronics laboratory, AIST Data Analysis Provision of MEG
Provision of expertise in the analysis of brain function Cybermedia
Center Data Generation Analysis Results Virtual Laboratory for
medicine and brain science Knowledge sharing MEG sharing? Data
Sharing
Slide 16
Requirements Computational and data intensive problem The
number of MEG instruments available is small. Knowledge of
scientists is distributed. No database? Different group uses
different analysis methods for different data.. Many medical
institutions and hospitals have no computers and that can satisfy
doctors analysis demand.
Slide 17
Wavelet cross-correlation analysis This analysis procedure
needs to be performed for each pair of MEG sensors. E.g. 64ch ->
2016 Raw MEG Data t f t f At 1 st Phase, wavelet transform Sensor A
Sensor B This image indicates that a brain signal with frequency f
was detected earlier in Sensor B than in Sensor A. f t At 2 nd
Phase, Wavelet cross-correlation f
Slide 18
Life-electronics laboratory, AIST Data Analysis Data Generation
Grid Resource Broker (Nimrod-G+Gridbus) 64 sensors MEG Results
World-Wide Grid [deadline, budget, optimization preference] 1 5 4 3
2 New Approach: Users QoS Requirements driven MEG Data Analysis on
the Grid Provision of MEG analysis Analysis All pairs (64x64) of
MEG data by shifting the temporal region of MEG data over time: 0
to 29750: 64x64x29750 jobs
Slide 19
Grid Enabling MEG data analysis Nature fine-grained jobs small
data sets Data Sets on Source Node High Latency for small jobs
Lower Efficiency Hence, data sets were replicated on each node
Application changed to access local datasets./metameg-datapath
time_offset time_offset_step meg_sensors_count Meg_data_path Output
is collated at the source node and then visualized Grid Enabled in
very short time ~ 1 week
Slide 20
Conclusion Introduced Gridbus Resource Broker using Application
Service Cost Described the Scheduling Algorithms followed Presented
Case Study of Brain Activity Analysis using our Resource Broker
Future Work: Integration with Accounting Mechanisms such as
GridBank Support for Group Scheduling and Economic-based Advance
Reservation of Resources