From Distributed to Pervasive OSGi Jan S. Rellermeyer Systems Group, ETH Zürich
From Distributed to Pervasive OSGi
Jan S. RellermeyerSystems Group, ETH Zürich
Welcome to Zürich!
Tuesday, June 23, 2009 2Jan S. Rellermeyer, ETH Zürich
Software Modules
Structured Programming Encapsulation Information Hiding Coupling vs. Cohesion Separation of Concerns
Reuse Orchestration
Software Design Principle Base unit for adding orthogonal concernsTuesday, June 23, 2009 3Jan S. Rellermeyer, ETH Zürich
David Parnas, 1972
Larry Constantine, 1974
Edsger W. Dijkstra, 1974
Web services, OSGi
The Future?
Module Management in Java
Traditional Java: The mystical class path
java –cp commons-X.jar foo.jar bar.jar my.application.MainClass
Which module contains the main class? What are the dependencies between foo.jar and bar.jar? What happens if bar.jar is upgraded to bar-1.01.jar?
Jan S. Rellermeyer, ETH ZürichTuesday, June 23, 2009 4
The OSGi Framework
OSGi Framework
Bundle A
Package 1Package 2Package 3
Bundle B
Package IPackage II
Exported Package Exported Package
Private Package
Import
5Jan S. Rellermeyer, ETH Zürich
Export-Package: Package 1 Export-Package: Package I, Package II
Import-Package: Package 1
Tuesday, June 23, 2009
Explicit dependencies Explicit, yet declarative Runtime
Dynamic Events, Introspection
Modularity: Coupling vs. Cohesion
Tuesday, June 23, 2009 6Jan S. Rellermeyer, ETH Zürich
Two components are loosely coupled, when changes in one never or rarely necessitate a change in the other
A component exhibits high cohesion when all its functions/methods are strongly related in terms of function
OSGi Services: Reducing Coupling
Bundles are Modules encapsulate functionality deployment unit
Enable reuse, extension, and dynamic composition
But: Package dependencies are explicit. Can lead to all or nothing Limits the modularity!
Solution: Services Separate the interface from the implementation
Interface
Implementation
Jan S. Rellermeyer, ETH ZürichTuesday, June 23, 2009 7
The Service Registry
The OSGi framework maintains a central service registry
Bundles can register their own services and retrieve services provided by other bundles
Services can be registered with a set of properties Additional description of the service, can be used to model
constraints or do “best fit matching”
Services are identified by their nameRegistry
Jan S. Rellermeyer, ETH ZürichTuesday, June 23, 2009 8
Distributed OSGi
General idea: use services provided by a remote machine Loose coupling
Remote can be Separate machine, connected through a network Separate JVM, different address space
(Different language, different address space)
Why would you want this? Isolation Redundancy Problem is inherently distributed
Tuesday, June 23, 2009 9Jan S. Rellermeyer, ETH Zürich
Preview: OSGi 4.2 Remote Services
Service Hooks Distribution Software can intercept service requests
Proxies Import a service into the local framework
Intents Denotes an abstract distribution capability Requires mutual agreement Derived from SCA
Tuesday, June 23, 2009 10Jan S. Rellermeyer, ETH Zürich
Example: Eclipse Communication Framework
OSGi/Equinox
Datashare
DiscoveryRemote Services
File Transfer
Presence Shared Object
CallShared Editing
XMPP (e.g.) ECF Core
IAdaptable
Container Adapters
Jingle
API Provider
container21
3
Datashare1
Application Eclipse, RCP, Equinox Server
Tuesday, June 23, 2009 11Jan S. Rellermeyer, ETH Zürich
ECF Remote Services
Can do RFC 119 Remote Service Provider Will be made compliant with the 4.2 specs Can use different distribution systems as backend
R-OSGi ECF generic DSOs
Can do more Proxy service has a distinct property service.imported set In ECF, this is set to an IRemoteService instance
One-Shot, fireAsync Futures, callAsync/1 Async with Listener, callAsync/2
Non-transparent access
Tuesday, June 23, 2009 12Jan S. Rellermeyer, ETH Zürich
R-OSGi
Preserves the expressiveness of OSGi Every Java class can be a service
Runs with every OSGi framework
Consistently maps network failures to module lifecycle events Proxy service provided by proxy bundle
Life-cycle, consistent behavior
Is flexible, yet competitive in terms of performance
Tuesday, June 23, 2009 13Jan S. Rellermeyer, ETH Zürich
[J. S. Rellermeyer, G. Alonso, T. Roscoe: R- OSGi: Distributed Applications through Software Modularization. In: Middleware 2007]
Dynamic Proxy Generation
Service: Interface + Implementation Shared Knowledge: Interface
public interface MyService {
public String callMe(Integer i);
}
public interface MyService {
public String callMe(Integer i);
}
public class MyServiceProxy implements MyService {
public String callMe(Integer i) {// generic remote service call
}
}
Peer A Peer B
Tuesday, June 23, 2009 14Jan S. Rellermeyer, ETH Zürich
Remote Services: Cohesion Distributed
Tuesday, June 23, 2009 15Jan S. Rellermeyer, ETH Zürich
Type Injection: Dealing with Coupling
Tuesday, June 23, 2009 16Jan S. Rellermeyer, ETH Zürich
public interface MyService {
void enqueue(QueueItem item);
Queue getQueue();
}
Bundle
MyService
Proxy Bundle
MyService
DISTRIBUTED OSGI
Location-transparency for servicesPoint to point
Not a single system imageNot a distributed module runtime
Tuesday, June 23, 2009 17Jan S. Rellermeyer, ETH Zürich
PERVASIVE OSGI
The Cloud(s)
Tuesday, June 23, 2009 18Jan S. Rellermeyer, ETH Zürich
Yahoo PipesAmazon EC2
Infrastructure as a service Pay as you go Horizontal Scale out
Agility Permanently available
The Fabric of the Cloud: Distributed Systems
Under the hood:
Potentially unreliable hardware unreliable network
Virtualized environment Little to no control
Tuesday, June 23, 2009 19Jan S. Rellermeyer, ETH Zürich
Distributed Systems Are Still Challenging
Architecture Complexity Parallelization Debugging, Testing Deployment Management
Tuesday, June 23, 2009 20Jan S. Rellermeyer, ETH Zürich
Software Modules as an Application Model
Composable Reusable Manageable Focus on functional aspects Encourage to think about interfaces Tool support
Building distributed systems without thinking of distribution
Think of R-OSGi
Tuesday, June 23, 2009 21Jan S. Rellermeyer, ETH Zürich
R-OSGi: A First Step Into The Cloud
Services facilitate loose coupling and high cohesionInstrumental for parallelization, think about Map-Reduce!
R-OSGi preserves the expressiveness of OSGi services Every Java class can be a service With R-OSGi, almost every OSGi service can be a remote service
Generality
Provides location transparency“Where” has a different meaning in a data center
Consistently maps network failures to module lifecycle events
Node and network failures happen often!
Tuesday, June 23, 2009 22Jan S. Rellermeyer, ETH Zürich
[J. S. Rellermeyer, M. Duller, G. Alonso: Engineering the Cloud from Software Modules. In: ICSE-Cloud 2009]
Example: R-OSGi Deployment Tool
Tuesday, June 23, 2009 23Jan S. Rellermeyer, ETH Zürich
[J. S. Rellermeyer, G. Alonso, T. Roscoe: Ready for Distribution? Turning Modular into Distributed Applications with the R- OSGi Deployment Tool . Demo at OOPSLA 2007]
Modules for the Cloud
Distribution can be added as an orthogonal concernR-OSGi turns OSGi modules into a distributed system
So can replication Instrumentation of the modules Middleware support
Elasticity through redundancy and redeployment
Tuesday, June 23, 2009 24Jan S. Rellermeyer, ETH Zürich
Module
Module
Module
Module
Module
Research Prototype: Cirrostratus
Tuesday, June 23, 2009 25Jan S. Rellermeyer, ETH Zürich
Overlay Network
A Virtual Runtime for Modules
“Hypovisor” for OSGi Modules
Instruments Bundles For paravirtualization For state capturing
Shares internal state Shared service registry Shared bundle registry
Shares bundle state Replication, migration
Tuesday, June 23, 2009 26Jan S. Rellermeyer, ETH Zürich
The Model of State
The fields of a service instance are “state” The fields of “state” are “state” “State” is replicated (and treated as “by reference”) Everything else is local to the node (no state leakage)
Tuesday, June 23, 2009 27Jan S. Rellermeyer, ETH Zürich
Code Analysis: Symbolic Executionadd(I)I
L0ALOAD 0DUPGETFIELD test/Simple.state : IILOAD 1IADDPUTFIELD test/Simple.state : I
L1ALOAD 0GETFIELD test/Simple.state : IIRETURN
L2LOCALVARIABLE this Ltest/Simple; L0 L2 0LOCALVARIABLE i I L0 L2 1MAXSTACK = 3MAXLOCALS = 2
Initialize the slots of the call stack with symbols 0 = “this”, 1 = arg0, 2 = local0
Initialize the fields with relative symbols test/Simple.state = “Simple.state”
Execute the code symbolically Execution stack becomes symbolic Non-linear control flow leads to Phi-
SymbolsTuesday, June 23, 2009 28Jan S. Rellermeyer, ETH Zürich
private int state;
int add(int i) {state += i;return state;
}
Instrumentation for Replication
private int state;
int add(int i) {state += i;return state;
}
private int state;
int add(int i) {TransactionContext.BOT(“Simple.add”, i);TransactionContext.read(“Simple.state”);state += i;TransactionContext.write(“Simple.state”, i);return state;TransactionContext.EOT();
}
Tuesday, June 23, 2009 29Jan S. Rellermeyer, ETH Zurich
Simplication, it’s top of stack
TransactionContext binds free variables at runtime and puts fields into context
Similar: Instrumentation for thread migration
A Virtual Module Runtime
Tuesday, June 23, 2009 30Jan S. Rellermeyer, ETH Zürich
+ non-functional requirements+ orthogonal concerns
CONCLUSIONS
OSGi is a very interesting platform for building dynamic modular applications for Java. Distribution software like R-OSGi facilitate remote access to OSGi
services. The 4.2 specs will bring this to the mainstream. The OSGi model is a perfect match for dynamic environments such as
cloud computing The Cirrostratus prototype enables adding even more sophisticated
properties to modules than distribution, such as state replication, service migration, hardening policies. It can thereby facilitate the even more pervasive usage of OSGi.
Questions?Tuesday, June 23, 2009 31Jan S. Rellermeyer, ETH Zürich