1 Configuring the Cloud Automated Planning as a Semantic Technology Blazej Bulka, PhD Clark & Parsia, LLC [email protected] www.clarkparsia.com Stuart Charlton Elastra [email protected] www.elastra.com 2010 Semantic Technology Conference
May 11, 2015
1
Configuring the CloudAutomated Planning as a Semantic
Technology
Blazej Bulka, PhDClark & Parsia, LLC
Stuart CharltonElastra
2010 Semantic TechnologyConference
Who we are?
Clark & Parsia is a semantic software startup
Offices in DC and Cambridge, MA
Software products for end-user and OEM use
Provides software development and integration services
Specializing in Semantic Web, web services, and advanced AI technologies for federal and enterprise customers
Elastra is a cloud computing software startup
Offices in San Francisco, CA
Co-Funded by Amazon.com, Bay Partners, and Hummer Winblad
Provides software and services for helping organizations migrate and manage their applications in private and public clouds
Elastra Cloud Server available for Amazon Web Services and VMWare
3
Outline
Introduction to automated planning Examples of planning systems Hierarchical planning Case Study: Planning in Cloud Computing Semantic technologies in planning:
HotPlanner from Clark & Parsia Conclusion
4
Introduction to automated planning
5
What is automated planning?
Automated process to determine which actions need to be taken to achieve a desired goal
Current state of the world Description of actions Goals and constraints
Planning system
Required actions (plan)
Multiple applications of produced plans− Software execution− Human execution− Assistance in solving a problem
6
Simple exampleInstalling new software package
Current state of the world: Description of installed software packages and their dependencies
Available actions:
− Retrieve list of packages available for install from a server− Retrieve package dependencies− Download software package− Install software package
Goal: Install software package X
Plan – sequence of actions specifying which packages to download and install, and in which order
7
Domain-dependent planning
Solves one type of problem well Specialized algorithm and data representation
(the designer encoded the structure of the problem and solution in the code)
Small modifications to problem (e.g., new kinds of dependencies between software packages) = modifications to the planning system
Larger modifications to problem = rewriting significant portions of the planning system
8
Domain-independent planning Solves problems in multiple, entirely different
domains (e.g., software management, truck routing, space mission control)
Domain (actions, state of the world, goals) have to be specified in a way understandable to the planning system
Generic planning algorithm Modifications to planning problem (small or
large) – modifications to the domain specification = No change of planning system's code
9
What makes a plan good?
Depends on the application Typical quality metrics
− Plan length (number of actions)− Makespan (time to execute the plan)− Plan cost (every action has a cost associated
with it)− Multi-objective metrics
10
Examples of planning systems
11
Space Exploration:Deep Space Network
Deep Space 1 (DS1)− First ever close pictures of a comet
(Borelly in 1999)
− Part of Deep Space Network
− Proof-of-concept for later systems
Benefits− Spacecraft autonomy
− Commands = goals
− Plan verification
− Execution monitoring
Images: nmp.nasa.gov/ds1/
12
Earth Observing-1 Mission (EO-1)
Proof-of-concept for autonomous sensor web (incl. satellites, buoys etc.)
Autonomous analysis of data
Feature detection (e.g., fire) and decide to investigate on its own
Collaboration among multiple satellites
Plan activities: transmissions, power usage, set-up, camera usage
Dramatic increase in amount of scientifically usable data (1000 x ?)
Images: eo1.gsfc.nasa.gov
13
Mars Exploration Rovers and MAPGEN
Autonomous execution of plans prepared on Earth
Very limited planning on Mars
MAPGEN – computer planning system assisting human planners
Can produce explanations of dependencies
Successor of MAPGEN is EUROPA (Extendable Uniform Remote Operations Planning Architecture)
Images: www.nasa.gov
14
Xerox RMP
Planning system for Rack Mounted Printing prototype
− System with multiple parallel printing engines (for performance and fault tolerance)
− Sheet path planning (and re-planning in case of failure)
Images: Do, Ruml, and Zhou (ICAPS 2008 Proceedings)
15
Planning in games
Strategy games and first-person shooters− Killzone 2 (Game of the Year 2009 in
Shooter category) 500 plans per second
− F.E.A.R. and Condemned: Criminal Origins
1616
1717
18
Siemens Tecnomatix 9CAM-oriented planning tasks
19
Hierarchical planning
20
Hierarchical planning
HTN (Hierarchical Task Network) planning Hierarchy of actions/goals Hint on the problem structure from the domain expert Prevents reinventing the wheel by the planning system Planner can first solve high-level goals and then refine
lower-level ones Significant improvement in performance Designed templates of desired solutions (e.g., plans
that adhere to internal policies)
21
Example HTN Plan
Goal: Have package X
Install package X
Download package list
Satisfy dependencies
for X
Goal: Have package X
Download X
InstallX
Have package Y
Have package Z
Install package Y
Satisfy dependencies
for Y
Download Y
InstallY
Already installed
22
Case Study: Configuring the CloudWith Elastra Enterprise Cloud Server
Elastra Enterprise Cloud Server
• Configure, Deploy, and Scale Multi-Tier Applications» Commercial and/or open source software
• Manage Applications Across a Hybrid Cloud» VMWare, Amazon Web Services, others
• Portable & Consistent Cloud Application Management
Federated Identity & Access Control
Application-Level Automation OS-Level AutomationMonitoring & Capacity
EventCorrelation Systems
Monitoring Systems
Configuration Agents
VirtualizationManagers
Public Cloud APIs3rd Party IT Systems
Integrate
DiscoverManageIntegratePublish
Subscribe
Automated Application Deployment & Scale OutPlanning & Orchestration
PDP PEP
24
The Elastic Modeling Languages:OWL Ontologies for Cloud Computing
Meeting the Challenges of IT Automation:How can these two servers communicate?
Possible areas of problems:• Security
» Bad credentials• Server Configuration
» Wrong IP or Port» Bad setup to listen or call
• Network Configuration» Wrong duplex» Bad DNS or DHCP
• Firewall Configuration» Ports or protocols not open
How do we usually automate a configuration?
• Scripts & Runbooks
• In Situation X, do this• In Situation Y, do this• In Situation Y, but exception Z, do this• ….• Problems: Context, Variability, Timing & Transitions• Scripts require you to explicitly write out each control
transition, assume you know in advance the one right way to do things
• That’s fine in the small; in the large, it gets complicated.
26
Elastra Next Generation Automation Engine:
Elastra Plan Composer
HTN Strategies
Strategy Examples:Configure Full SystemScale-Out a SystemRecover a Database
HTN TacticsSearch for Next Tactics :Set Firewall RulesRestore FilesystemInstall & Start ServiceProvision Network
HTN AtomicActions
Search for Atomic Bindings:Configuration Agent CalloutCloud API CalloutVirtualization Manager Callout
Application& Deployment
Data
(BackgroundIndividuals)
ElasticModeling
Languages(OWL v2 Ontology)
Clark & Parsia HotPlanner
Clark & Parsia Pellet
Jena + SPARQL
27
Elastra’s Success Results with HotPlanner
• Reduced, More Maintainable Codebase» Custom SPARQL and Java Code to interpret RDF, vs.
HotPlanner’s Domain-Specific Language for HTN» Up to a 3x reduction in code length for various modules
• Performance» Ability to generate a plan in under a minute for a 20 server, 20
component, multi-tier system design» Typical planning time: 3 to 5 seconds, up to 15 for complex» Near-equivalent to classic manual SPARQL-based analyzer
• Visibility and Modifiability of Plan» Plans are just OWL-S processes, easy to parse, render to GUI,
and/or modify before execution
28
29
Semantic Technologies in Planning
30
Semantic technologies in planning
Semantic technologies have not been used together with planning frequently so far
− Promising opportunity Modern semantic technologies (RDF and
OWL) are relatively new compared to planning Planning finally became usable in real-world
settings− The two areas can contribute to each other− Many places where one area's weakness is
the other's strength
31
Expressing planning domain
Planners depend on accurate formal description of planning domain
− Language to describe the current state of the world
− Goals
− Actions Parameters (e.g., install package X) Preconditions (when an action may be executed) Effects (what changes in the world when the action is executed) Hierarchy
Ideal solution− Expressive (e.g., describing quantities)
− Keep computational complexity under control
32
Classical planning formalisms are inadequate
Traditional formalisms− STRIPS, PDDL (Planning Domain Definition
Language)− Limited expressivity
Expressing complex domains difficult− Support for hierarchical planning is sparse
33
Representing planning knowledge in OWL
Describing the current state of the world in RDF and OWL
− Expressive language− Designed with computational complexity in
mind (e.g., various profiles EL, RL, QL)− Can be used to infer information about the
current state of the world that is not explicitly specified
34
Describing actions and hierarchies based on OWL-S
OWL-S is designed to specify descriptions of Web services
Can be used to specify any action in Hierarchical Planning by using owls:Process
− Composition of actions (hierarchy)− Preconditions− Effects− Parameters of actions
35
Challenges
OWL ontologies are designed to represent static data
Planning includes reasoning about changes− Describing effects of action (some axioms
become true, other become false)− Identifying invariants that are preserved− Matching effects of one actions with
preconditions of others
36
HotPlanner Clark & Parsia's response to these challenges Used in production systems Uses Pellet OWL reasoner for state
knowledge base management, reasoning and query answering– Can plan and optimize for multiple objectives
– Extension to OWL-S for actions
– Scheduling and planning for parallel task execution
For more information go to http://clarkparsia.com/planner
37
Conclusions
Planning is a proven technology for task automation, decision support, autonomous systems
Semantic technologies and planning have a lot to offer to each other
Automated planning is quickly growing in use for Cloud Computing automation
38
Thank you for your attention!Questions?