Denis Caromel 1 D. Caromel , et al. INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF June 23 2005, Beijing Open Source Middleware for the Grid: ObjectWeb ProActive 1. Asynchronous Distributed Objects: ProActive 2. Example of Application: 3D Electromagnetism 3. Composing for the Grids: Components ProActive. ObjectWeb. org
57
Embed
Denis Caromel1 D. Caromel, et al. INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF June 23 2005, Beijing Open Source Middleware for the Grid:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Denis Caromel 1
D. Caromel, et al. INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF
June 23 2005, Beijing
Open Source Middleware for the Grid: ObjectWeb ProActive
1. Asynchronous Distributed Objects: ProActive
2. Example of Application: 3D Electromagnetism
3. Composing for the Grids: Components
ProActive.
ObjectWeb.
org
Denis Caromel 2
Grid ComputingLos Angeles
Beijing
Amsterdam
Sophia Antipolis
Challenges: Programming Model,
Scale, Latency, Heterogeneity,
Versatility (protocols, firewalls, etc.)
Denis Caromel 3
1. Distributed Objects
P r o g r a m m i n g
Composing
Deploying
W r a p p i n g
Denis Caromel 4
A uniform framework: An Active Object pattern A formal model behind: Determinism (POPL’04)
Programming Model:• Remote Objects• Asynchronous Communications, Wait-By-Necessity • Groups, Mobility, Components, Security, Fault-ToleranceEnvironment:• XML Deployment Descriptors• Interfaced with: rsh, ssh, LSF, PBS, Globus, Jini, SUN Grid Engine• Graphical Visualization and monitoring: IC2DIn the www. ObjectWeb .org Consortium (Open Source LGPL)
ProActive: A Java API + Tools for Parallel, Distributed Computing
Denis Caromel 5
A
ProActive : Active objects
Proxy
Java Object
A ag = newActive (“A”, […], VirtualNode)V v1 = ag.foo (param);V v2 = ag.bar (param);...v1.bar(); //Wait-By-Necessity
V
Wait-By-Necessity
is a
Dataflow
Synchronization
JVM
A
JVM
Active Object
Future Object Request
Req. Queue
Thread
v1v2 ag
WBN!
Denis Caromel 6
Explicit Synchronizations
Single Future Synchronization:ProActive.isAwaited (v); // Test if available .waitFor (v); // Wait if not available
Vectors of Futures: .waitForAll (Vector); // Wait all of them .waitForAny (Vector); // Get One
A ag = newActive (“A”, […], VirtualNode)V v = ag.foo(param);... v.bar(); // Wait-by-necessity
Denis Caromel 7
A
Creating AO and Groups
Typed Group Java or Active Object
A ag = newActiveGroup (“A”, […], VirtualNode)V v = ag.foo(param);...v.bar(); //Wait-by-necessity
V
Group, Type, and Asynchrony
are crucial for Cpt. and GRID
JVM
Object-Oriented
Typed Group Communications
Denis Caromel 8
OO SPMD A ag = newSPMDGroup (“A”, […], VirtualNode)
// In each member myGroup.barrier (“2D”); // Global Barrier myGroup.barrier (“vertical”); // Any Barrier myGroup.barrier (“north”,”south”,“east”,“west”);
A
Still,
not based on raw messages, but
Typed Method Calls
==> Components
Denis Caromel 9
IC2D: Interactive Control and Debugging of Distribution
With any ProActive applicationFeatures:
Graphical and Textual visualization Monitoring and Control
Denis Caromel 10
Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore
ProActive IC2D:
Width of links
proportional
to the number
of com-
munications
Denis Caromel 11
2. Application
3D Electromagnetism
Denis Caromel 12
JEM 3D : Java 3D Electromagnetism
Maxwell 3D equation solver, Finite Volume Method (FVM)Pre-existing Fortran MPI version: EM3D (CAIMAN team @ INRIA)
Execution Time on a cluster
0
100
200
300
400
500
600
700
800
900
0 10 20 30 40 50 60 70nombre de processeurs
tem
ps
(sec
on
des
)
21*21*21
31*31*31
43*43*43
55*55*55
81*81*81
97*97*97
113*113*113
121*121*121
Mesh Size
Denis Caromel 13
Interface
Denis Caromel 14
Interface
Denis Caromel 15
Interface
Denis Caromel 16
Interface
Denis Caromel 17
Beating Fortran MPI ?
Current status:
• Sequential Java vs. Fortran code: 2 times slower
• Large data sets in Java ProActive: 150x150x150 (100 million facets)
• Large number of machines: up to 294 machines in Desktop P2P• Speed up on 16 machines:- Fortran: 13.8
- ProActive/Ibis: 12
- ProActive/RMI:8.8
Grid on 5 clusters (DAS II, Netherlands): Speed up of 100 on 150 machines
Fortran: no more than 40 proc. …
Beating Fortran MPI with Java ProActive? X/40 (14/16) = 2X/ n (100/150)
Yes, starting at 105 machines !
Denis Caromel 18
3. Components
for
The GRIDs
Denis Caromel 19
Content
Controller
The Fractal model:Hierarchical Components
Common component model of the ObjectWeb consortium
Denis Caromel 20
Content
Controller
Interfaces = Provided and Required
Provided,
Server
Interfaces
Required,
Client
Interfaces
Denis Caromel 21
Content
Controller
Hierarchical model : Composites encapsulate Primitives,
Primitives encapsulate Code
Denis Caromel 22
Content
Controller
Binding = in an external file (XML ADL), Not in programs
Denis Caromel 23
Content
Controller
Binding = in an external file (XML ADL), Not in programs
Denis Caromel 24
Graphical Interface for Composing Components
Denis Caromel 25
ProActive Component DefinitionA component is:
• Formed from one (or several) Active Object
• Executing on one (or several) JVM
• Provides a set of server ports: Java Interfaces
• Uses a set of client ports: Java Attributes
• Point-to-point or Group communication between components
Hierarchical:• Primitive component: define with Java code and a descriptor
• Composite component: composition of primitive + composite
• Parallel component: multicast of calls in composites
Descriptor:• XML definition of primitive and composite (ADL)
• Virtual nodes capture the deployment capacities and needs
Virtual Node is a very important abstraction for GRID components
Denis Caromel 26
Objects to Distributed Components
Typed Group Java or Active Object
ComponentIdentity Cpt = newActiveComponent (params);A a = Cpt … .getFcInterface ("interfaceName");V v = a.foo(param);
V
AExample of
component
instance
JVM
Denis Caromel 27
A
A
B
C
P
Group proxyGroup proxy
A
B
C
D
Groups in Components
Broadcast at binding,
on client interfaceAt composition,
on composite inner server interface
A parallel component!
Denis Caromel 28
Wrapping Legacy MPI Components
MPI CodeC/Fortran:
Messages on Tags
sent/converted to
Method Calls
ProActive Java:
Method Calls
sent as
Messages on Tags
Virtual Nodes for Deployments
Denis Caromel 29
On-going : MxN communicationsControl at binding points
M components
N components
GATHERING
SCATTERING
REDISTRIBUTION from M to N also, Functional Code
Denis Caromel 30
Call For ContributionsGCM: Grid Component Model
Within CoreGRID
In charge (M. Danelutto, D. Caromel) of defining a Generic, Comprehensive, Open, European and World Wide
GCM: Grid Component Model
Ways to participate: Email to us, come to the GRID @ Work Workshop Oct. 14 2005
Denis Caromel 31
Conclusions and A Few Directions
ProActive: A Strong Programming Model + Components
FACTS AND FIGURES
June 10 World Record: 52-years computation in 6 months in Desktop P2P
Deployed at once on 1000 CPUs (Plugtests on ssh, Globus, LSF, ...)
(Close to) Beating Fortran on an Electromagnetic Application
Looking for collaborations: • Building reusable Cpts from Numerical Codes • Generic Techniques for Wrapping Codes
Available in LGPL with ObjectWeb
http://ProActive.ObjectWeb.org
Denis Caromel 32
Denis Caromel 33
ProActive Non Functional Properties
Currently in ProActive:
• Remotely accessible Objects(Classes, not only Interfaces, Dynamic)