Virtual Playground: from Quality of Service to Quality of Life in the Grids Katarzyna Keahey Argonne National Laboratory.

Virtual Playground: from Quality of Service

to Quality of Life in the Grids

Katarzyna Keahey

Argonne National Laboratory

06/26/04 Katarzyna Keahey: PSNC Seminar

What are computational Grids?

They are like power Grids: remote computing power can be accessible from your laptop in much the same way as electricity is delivered to your home


Are they really? If power grids were like computational grids…

A simple task, like making a toast: Where is/are your power station(s)? How much power do they produce? Does the power supply pattern vary through the day? It is not clear who would be charged for your toast… What if your power station goes down?

Unless you can handle these questions… no toast! We need:

Quality of Service: reliable access to required amount of computational power when you need it

Quality of Protection: data secure in predictable ways Quality of X: you name it Last but not least: Quality of Life!


Enter Agreement-Based Interactions

Traditional Resource Management Model Assumption of global ownership Provider-centric: utilization as primary metric Closed System

Dynamic Policy-driven Resource Management Resources in a Global Context

Advertisement and dynamic negotiation/renegotiation Resource maintains autonomy

Clients or Agents Negotiate for Resources WS-Agreement

Dynamic, decentralized, policy creation and adjustment Abstracts impact of other policies and users

Operating policies may be proprietary, global knowledge is hard to achieve, etc.


What is WS-Agreement? An agreement defines a dynamically-established and

dynamically-managed relationship between parties. The object of the relationship is the delivery of a service by one of the parties within the context of the agreement. The management of this delivery is achieved by agreeing on the respective roles, rights and obligations of the parties. The agreement may specify not only functional properties for identification or creation of the service, but also non-functional properties of the service such as performance or availability. Entities can dynamically establish and manage agreements via Web service interfaces.

How is agreement different from policy? Policy is static Agreements are dynamic: they provide mechanisms and protocols for

creating and changing policies on the fly Agreement may imply an action

Information about the action enters the system as well as a policy allowing that action


WS-Agreement (cntd) What do agreements look like?

Agreements will often be finer-grained than policies Agreements may represent an ephemeral or periodic

policy They may involve “late binding” There may be a large volume of agreements

Other agreement properties Simple way of expressing the aggregate of policies in the

system May hide proprietary policies from the user Allow providers to gauge demand Establish policies to reconcile conflicting terms and goals

Client: deadlines and availability goals Applications: need coordinated resources Resource owners: localized utilization goals, satisfying demand


WS-Agreement Model Negotiation Layer

For managing Agreements

Complex dialogues Coordination

Agreement Layer Externalized policy For managing service

providers Service Layer

Policy State Affects service

Domain-specific

Application Instance

FactoryPolicycreate()

foo()

create()Agreement

Ops:terminate(limits)inspect(query)...

SDEs:

Terms RelatedStatusAgrmts.

inspect()

Factory

Consumer Provider

Manager

Negotiation

Ops:terminate(limits)negotiate(...)...

SDEs:

Terms RelatedStatusAgrmts.

Factorycreate()

negotiate()

Negotiator


Agreement Abstractions Agreement Factory

Persistent service For example: façade to scheduler(s) Advertises agreement template

Advertises what provider has to offer (contraints/offers) Creates Agreements

Agreement Transient resource Allows inspection of terms For example: job entry virtualized into a service Lifetime maps to lifetime of “terms of service” Allows the client to renegotiate


Agreement Negotiation

Agreement template -> acceptable agreement Phases:

Negotiation admission Iteration Acceptance Commitment

Fine-grained versus coarse-grained negotiation Term-level versus agreement-level Compromise between practicality and efficiency

Commitment models


Claiming Agreements

Claiming: associate an agreement with an entity apply_agreement(A,X); Considered domain-specific

Different claiming models No claiming required: the actions to be taken are

sufficiently well described by the agreement Event-based claiming Late binding

“apply this resource reservation to this execution”

Disassociate agreements Bind an agreement to a different entity


Challenges Synchronization

Commitment When does a renegotiated agreement replace the current one?

Security and ownership Transferability Chaining agreements Can agreements be tokens Security roles

Agreement Terms Composing agreements Directionality Other concerns

Tools and Usability Performance


The National Fusion Collaboratory:Using Agreements for Science

paper: Grids for Experimental Science: the Virtual Control Room CLADE 2004


Shared Application Real Time Data Display

Video & Audio Between Pulse Data

Shot Cycle Status

FusionGrid

Remote control room experimental “mock up” @ SC’03

November, 2003


Grid service Usage (NFC) Agreements for between-pulse execution

Dynamically created application-level agreements “whenever I run a certain configuration of EFIT tomorrow between 9am and 5pm I

want it to run as fast as possible and no longer than 4 minutes” Late binding

Agreement claiming is triggered by event: experimental data becomes available

E2E Agreements Job execution

Based on resource reservation and prediction of run-time Resource reservation Data Transfer

Based on prediction, resource management and adaptation Aggregating agreements paper: Agreement-Based Interactions for Experimental Science,

Europar 2004 Prototype implementation

Loosely compliant with the WS-Agreement spec


We Have an Agreement: Now What?

Provider’s role Resource management

Advance reservation, rate limiting, etc.

Prediction Is this really an agreement?

Adaptation All of the above!

Paper: Providing Data Transfer with QoS as Agreement-Based Service, SCC 2004

Resource management versus prediction Confidence levels and guarantees


Agreement Enforcement Effective agreement enforcement

Q: How can we enforce agreements? A: Execution environment with QoS enforcement

Aspects of an execution environment Configurable, virtual resource

Enforcement of CPU, disk memory usage Sandboxes, virtual machines, etc.

Software configuration Defined environment in terms of user-defined qualities: files, library

signature, etc.

Dynamically, flexibly, remotely, automatically configurable…

To reflect potentially changing nature of agreements and policies in the Grid


Virtual Workspaces

Virtual resource configuration

Protection environment

Software and file configuration state

Execution state

Virt

ual W

orks

pace

Grid

Mid

dle

wa

re

Inte

rfa

ceG

rid c

lien

t In

terf

ace

Grid clients

Grid middlewareinterface

Define interfaces and explore a variety of implementations Describe a VW using an ontology

Properties of an implementation (e.g. how to deploy it on the Grid)

Content of the workspace


Properties Virtual Workspaces Dynamic creation

Inherent property of Grid computation Examples of existing mechanisms: single sign-on, GRAM

Avoiding a maintenance nightmare (automate administration) and potential security hazard

Dynamic configuration To reflect changing policies in the Grid (implement agreements)

Strong protection environment Otherwise users won’t trust sites and sites won’t trust users

Fine-grain enforcement Configurable architecture, software, environment

Application software/libraries/licenses Configurable environment Running 32-bit programs on 64-bit architectures Running a required version of the OS (Fedora vs. RH9) Potentially even execution state


Technologies

Unix accounts and Unix account tools setrlimit, DSRT, chroot, chown, etc.

Deployment & configuration tools Pacman & pacman cache

Sandboxes VServer, protection and fine-grain enforcement

Virtual Machines VMware, Xen


Interacting with VWs

Clie

nt

request

VW EPR

inspect and manage

deploy & suspend

use existing VW Create VW

VW Factory

VW Repository

VW Manager

create new VW

ResourceVW

start program


Services Factory

Creates VW in terms of the implementation e.g., VM image, pacman chache+

May create based on an already created VW Writes/configures access policy May allow negotiation

VW Repository Access to state describing a VW Allows inspection, management, implementation-specific

termination, potentially renegotiation, etc. Soft-state lifetime management ensures termination

VW Manager Lightweight infrastructure deploying VMs


Comparing VW Technologies Unix accounts

Pros: efficient, ubiquitous Cons: very limited functionality Needs to be used in conjunction with other technologies

Pacman, system tools

Prototype available (GT 3.2) http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm

Sandboxes Pros: efficient, fine-grain enforcement, typically very

lightweight Cons: limited state enforcement Need to be used in conjunction with other technologies

http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm

http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm


Comparing VW Technologies (cntd)

Virtual Machines Pros:

Flexibility (run linux on linux, 32 on 64-bit, etc.) Enhanced security, audit forensics, etc. Great user state management Freezing/migration Customized environment A promising distribution/deployment tool

Cons: Potential for being less efficient (emulation) Resource inefficient Poor implementation of sharing, relatively little enforcement

(but can be combined with other technologies for enforcement)

The potential is excellent, but needs more work


What about efficiency?

0

0.2

0.4

0.6

0.8

1

1.2

110100jt 110105jt 110109jt

UNIX acctVserverVMware

also see: Figuieredo et al, A case for Grid Computing on Virtual Machines


Other efficiency concerns

Startup time

Resource use efficiency VMware memory use: 24MB + 1 MB per 32 MB

memory allocated per VM Upcoming paper

Table 1: DVE create/destroy times

Linux VServer VMware

Create 100 ms 360 ms 14-52 sec

Destroy 70 ms 200 ms 3-38 sec


Toward Virtual Playground More than one sandbox = playground Managing VW-based execution in the Grids

A virtual Grid of interactions New potential creates new challenges

Networking issues VM networking issues VNET

Sharing issues How do we orchestrate sharing between VMs? We have not even defined what sharing really means!

How do we represent and manage a “playground” as a Grid session?


Conclusions Future Grids will rely on dynamic, decentralized,

policy-driven resource management Need for more flexibility QoS issues

Agreements are a promising solution to creating policy on the fly

There will be more emphasis on enforcement technologies

Virtual Workspaces offer a comprehensive hardware/software solution We enforce not only resource usage Virtual is the new real!

Virtual Playground: from Quality of Service to Quality of Life in the Grids Katarzyna Keahey Argonne National Laboratory.

Documents

system u

demand u

entity u

schedulers u

user u

contraintsoffers u

fly u agreement

u quality of service