Top Banner

Click here to load reader

of 20

Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis Srikumar Venugopal 1, Rajkumar.

Dec 26, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • Slide 1
  • Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis Srikumar Venugopal 1, Rajkumar Buyya 1, Susumu Date 2 1.Grid computing & Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australia www.gridbus.org/www.gridbus.org/ 2. Cybermedia Center, Osaka University GRIDS
  • Slide 2
  • What does a Resource Broker do? Gets user/application requirements Discovers resources like computational nodes, data sources, etc. Establishes costs, user credit, etc. Makes decisions about the optimal schedule for jobs Dispatches jobs Resource Broker Cataloguing Services Accounting Services Information Services Application Grid Nodes
  • Slide 3
  • Architecture of Gridbus Scheduler Data Catalogue ASP Catalogue Gridbus Scheduler Grid Info Service Grid Market Directory Grid Node GridBank Agent+Job Grid Node (e.g., UofM) PE Grid Node (e.g., VPAC) PE Grid Node (e.g., ANL) CPU or PE Access Service (Globus) Globus GTS Cluster Scheduler Application Results Bill Visual Parametric Tool
  • Slide 4
  • Gridbus Scheduler Interfaces with: Application Development - Visual Parametric Tool Information Services - Grid Market Directory (Cost), GRIS,etc. Accounting Services - Grid Trading Service, GridBank Cataloguing Services - Application Catalog, Replica Catalog Job Dispatcher Nimrod-G (for parametric jobs) Gridbus Dispatcher (for data intensive, reservation, P-GRADE support, etc.) work in progress Supports: User-specified QoS parameters such as Deadline, budget, etc. Application Cost or Hardware Cost (CPU, etc) Cost from Grid Market Directory or Flat File Cost, Time or Cost-Time Optimization.
  • Slide 5
  • Application Service Costs? Present Approach to Processing Cost - Timeshare or CPU cycles used Users more interested in the cost of getting job done than amount of processing power consumed New Approach to Cost - Application Service Costs charge for using the application once. Different costs for different applications depends on provider Broker finds Cost through Grid Market Directory.
  • Slide 6
  • Scheduling Algorithms Gridbus Scheduler implements Cost Optimization Minimize computational cost (within deadline) Time Optimization Minimize execution time (within budget) Cost-time Optimization Similar to cost-optimization Implemented for first time.
  • Slide 7
  • Scheduling (contd..) Uses past performance to forecast each machines capacity The rate of completion is averaged to compensate for any spikes or troughs Cost Optimization Gives maximum jobs to the cheapest machine Time Optimization Gives jobs to machines based on consumption rate but limited by budget per job Cost-Time Optimization Distributes jobs among the machines of consumption sorted by their consumption rate
  • Slide 8
  • Cost Optimization: No. of Jobs Done vs time
  • Slide 9
  • Cost-Time Optimization: No. of Jobs Done vs Time
  • Slide 10
  • Time Optimization: No. of Jobs Done vs Time
  • Slide 11
  • Comparison of Scheduling Algorithms All experiments were started with No of Jobs = 200 Deadline = 2hrs Budget = 600 Grid $ Start TimeCompletion TimeBudget Consumed (Grid $) Cost10:00 a.m.11:27 a.m.188 Cost-Time11:40 a.m.12:08 p.m.277 Time12:30 p.m.12:59 p.m.274
  • Slide 12
  • Case Study: Brain Activity Analysis In Collaboration with Osaka University, Japan Computationally and data intensive
  • Slide 13
  • MEG Data/Brain Activity Analysis MEG (Magnetoencephalography) Achieve both non-invasiveness and high degree of measurement accuracy cf. EEG (Electroencephalography), ECoG (Electrocorticography) Measure functional data on multiple points around the head Promising among medical doctors and brain scientists. A B A: B: http://www.ctf.com
  • Slide 14
  • MEG data analysis Osaka Univ. Hospital Osaka Univ. DV transfer Life-electronics laboratory, AIST Data Analysis Provision of MEG Provision of expertise in the analysis of brain function Cybermedia Center Data Generation Analysis Results
  • Slide 15
  • MEG data analysis Osaka Univ. Hospital Osaka Univ. DV transfer Life-electronics laboratory, AIST Data Analysis Provision of MEG Provision of expertise in the analysis of brain function Cybermedia Center Data Generation Analysis Results Virtual Laboratory for medicine and brain science Knowledge sharing MEG sharing? Data Sharing
  • Slide 16
  • Requirements Computational and data intensive problem The number of MEG instruments available is small. Knowledge of scientists is distributed. No database? Different group uses different analysis methods for different data.. Many medical institutions and hospitals have no computers and that can satisfy doctors analysis demand.
  • Slide 17
  • Wavelet cross-correlation analysis This analysis procedure needs to be performed for each pair of MEG sensors. E.g. 64ch -> 2016 Raw MEG Data t f t f At 1 st Phase, wavelet transform Sensor A Sensor B This image indicates that a brain signal with frequency f was detected earlier in Sensor B than in Sensor A. f t At 2 nd Phase, Wavelet cross-correlation f
  • Slide 18
  • Life-electronics laboratory, AIST Data Analysis Data Generation Grid Resource Broker (Nimrod-G+Gridbus) 64 sensors MEG Results World-Wide Grid [deadline, budget, optimization preference] 1 5 4 3 2 New Approach: Users QoS Requirements driven MEG Data Analysis on the Grid Provision of MEG analysis Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs
  • Slide 19
  • Grid Enabling MEG data analysis Nature fine-grained jobs small data sets Data Sets on Source Node High Latency for small jobs Lower Efficiency Hence, data sets were replicated on each node Application changed to access local datasets./metameg-datapath time_offset time_offset_step meg_sensors_count Meg_data_path Output is collated at the source node and then visualized Grid Enabled in very short time ~ 1 week
  • Slide 20
  • Conclusion Introduced Gridbus Resource Broker using Application Service Cost Described the Scheduling Algorithms followed Presented Case Study of Brain Activity Analysis using our Resource Broker Future Work: Integration with Accounting Mechanisms such as GridBank Support for Group Scheduling and Economic-based Advance Reservation of Resources