‹#›
OpenStack and Cloud Foundry!Pair the leading open source Infrastructure-as-a-Service!with the leading open source Platform-as-a-Service! Daniel Krook!Senior Certified IT Specialist, IBM!@danielkrook – krook.info
What do you want to build with the cloud?
Meet OpenStack and Cloud Foundry!
See why they’re a great choice for your next
project!
Learn how they can be
integrated to run your apps!
Join the community and
learn more!
Then maybe OpenStack and Cloud Foundry are the right tools for you
Cloud Computing e-Business
Service Oriented Architecture
Social Business
Mobile Computing
Open Cloud Architecture
The IBM Cloud is made with open source software
Daniel Krook – Senior Certified IT Specialist, IBM!@danielkrook – krook.info
IBM invested $1 billion in Linux It’s doing the same for OpenStack and Cloud Foundry
Cloud delivery models and implementations
Standardization; lower costs; faster time to value!
Networking!
Storage!
Servers!
Virtualization!
O/S!
Middleware!
Runtime!
Data!
Applications!
Traditional !on premise!
Networking!
Storage!
Servers!
Virtualization!
O/S!
Middleware!
Runtime!
Data!
Applications!
Networking!
Storage!
Servers!
Virtualization!
O/S!
Middleware!
Runtime!
Data!
Applications!
Networking!
Storage!
Servers!
Virtualization!
Middleware!
Runtime!
Data!
Applications!
O/S!
Vendor manages in cloud!Client manages!
API economy!Cloud operating environment!Software defined environments!
Infrastructure! Platform! Software!
Sys admin (Ops)! App builder (Dev)! Business user (Biz)!
IaaS: the software defined data center
Networking!
Storage!
Servers!
Virtualization!
Middleware!
Runtime!
Data!
Applications!
O/S!
Software defined environments!
Infrastructure!
Ops!
PaaS: the cloud operating environment
Networking!
Storage!
Servers!
Virtualization!
O/S!
Middleware!
Runtime!
Data!
Applications!
Cloud operating environment!
Platform!
Dev!
SaaS: the API economy
Networking!
Storage!
Servers!
Virtualization!
O/S!
Middleware!
Runtime!
Data!
Applications!
API economy!
Software!
Biz!
OAuth
The OpenStack mission is to produce the ubiquitous open source cloud computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.
OpenStack is designed around several core tenets • Simple to implement, massively scalable, elastic, and feature rich
• Architected to provide flexibility as you design your cloud
• No proprietary hardware or software requirements
• Able to integrate with legacy systems and third party technologies
• Share-nothing architecture – composable stand-alone services
• API driven and command line accessible
• Stateless and asynchronous
• Flexible networking models to suit the needs of different applications or user groups
OpenStack is composed of a core set of projects Compute (Nova) Provision and manage virtual machines
Dashboard (Horizon) Self-service portal
Image (Glance)Catalog and manage server images
Identity (Keystone)Unified authentication, integrates with existing systems
Object Storage (Swift)Scalable, secure, reliable object storage
Network (Neutron) Provides flexible networking-as-a-service
Block Storage (Cinder)Allows block devices to be exposed and connected to compute instances
And more… Metering (Ceilometer), Orchestration (Heat), DBaaS (Trove), MapReduce (Sahara)
Network Dashboard
Compute Image Object Storage
Identity Block Storage
Components collaborate via messaging and REST APIs
Accumulated Community Population
Accumulated Developer Population
Monthly number of participants
Monthly GIT contributors
Activity Community
Source: Qingye Jiang’s Blog:: CY14-Q1 Community Analysis — OpenStack vs OpenNebula vs Eucalyptus vs CloudStack-www.qyjohn.net/?p=3522
CURRENT CONTRIBUTORS AVERAGE MONTHLY CONTRIBUTORS
2,130 466
382
17,210 INDIVIDUAL MEMBERS
ORGANIZATIONS
PATCHES MERGED
17,209
COUNTRIES
139
OpenStack is the largest open source project, let alone IaaS
RELEASES Icehouse – 4/14 Havana – 10/13 Grizzly – 4/13 Folsom – 9/12
Essex – 4/12 Diablo – 9/11 Cactus – 4/11 Bexar – 2/11
Austin – 10/10
Only Platinum and Gold members are shown There are dozens of others
The OpenStack Foundation
The most recent OpenStack Summit attracted nearly 5,000 people
Proof point of OpenStack’s momentum & focus !§ 4,600+ in attendance (>30% growth) § 65% are new attendees to the Summit § Overall increase in product strategy roles § Voice of the user continues to shape software § Broad ecosystem support ensures rapid innovation § Focus on enterprise features in Icehouse and Juno accelerates maturity
A"endee Roles
Starting simple with three types of node
Controller Node
Compute Nodes Compute Nodes Compute Node
Nova Compute
Nova Network
Storage Node
Cinder (volume)
Glance (image)
1x
20x
Dedicated !Servers !
(bare metal) Nova API
Nova Scheduler
Keystone
Horizon
Database (MySQL)
Messaging (Qpid)
1x
Moving towards a resilient model
Compute Nodes Compute Nodes Compute Node
Nova Compute
Cloud Controller Data Node database (mysql)
messaging (qpid)
Cloud Controller Master Node
nova-api
nova-scheduler
Keystone
Horizon
Cloud Controller Data Node database (MySql)
messaging (Qpid)
Cloud Controller Storage Node
Cinder (volume)
Glance (image) Nova Network
Controller Node
Nova API
Nova Scheduler
Keystone
Horizon
Storage Node
Cinder (volume)
Glance (image)
2x Data Node
Database (MySQL)
Messaging (RabbitMQ)
2x 15x
Load Balancer Node HA Proxy
Keepalive
Load Balancer Node HA Proxy
Keepalive
Load Balancer Node HA Proxy
KeepAlive
6x 3x Virtual Machines!
(VMs)
Dedicated !Servers !
(bare metal)
Cloud Foundry is the industry’s Open PaaS and provides a choice of clouds, frameworks and application services. Its unique vision is to foster contributions from a broad community of developers, users, customers, partners and ISVs while advancing development of the platform at extreme velocity.
Cloud Foundry is built on a highly scalable distributed architecture
Routes incoming traffic to the appropriate component; generally the Cloud Controller or application
Identity management service for the platform. Acts as an OAuth2 and SCIM provider.
Monitors the state of applications and ensures that the correct number of instances are running
Exposes a REST API to the system. Manages a database of apps, services, service instances, etc
End user provided code that is “pushed” to the cloud and packaged to run in warden container
Advertises service offerings to the Cloud Controller and handles requests to create, bind, unbind, and delete service instances.
Managed by the service gateway, these nodes host service instances. A broker that can expose a service instance to a service managed outside of the cloud
A collection of code that is responsible for transforming pushed app artifacts into a ready to run droplet
A pub-sub message bus implemented with NATS that’s used for cross component communication
Droplet Execution Agent – Advertises capacity to execute droplets created by a build pack
A distributed deployment install and management tool that abstracts the details of the IaaS layer
Components collaborate via messaging and REST APIs
Buildpacks and services lend Cloud Foundry its extensibility Buildpacks bin/detect bin/compile bin/release bin/package
The detect script is used to determine whether or not to apply the buildpack to an application.
The compile script builds the droplet that will be run by the DEA and will therefore contain all the components necessary to run the application.
The release script provides feedback metadata back to Cloud Foundry indicating how the application should be executed.
The package script provides artifacts, which are provided to Cloud Foundry as system buildpacks. package is intended to provide a way for developers to package a buildpack with its dependencies.
But Cloud Foundry is more than just the code
Meets Developer’s Needs Focus on app development, not provisioning VMs, databases,
messaging servers, etc. Agile development model
Deploy and scale in seconds
Open Cloud Platform There is an increasing appetite for
cloud-based mobile, social and analytics applications from line-of-business executives - drives the
need for a more open cloud development platform
Compelling Community Cloud Foundry has a compelling
community and emerging ecosystem as well as a mature set of capabilities
and robustness
The Cloud Foundry Foundation
The second Cloud Foundry Summit attracted almost 1,000 people
• 5100+ mentions of #CFSummit, CF Summit & Cloud Foundry Summit!
• 3246 mentions of Cloud Foundry!• 5000+ mentions of #PaaS during Summit!
Cloud Foundry Summit Twi4er sen7ment
Summit growth signals dynamic ecosystem!§ 904 in attendance (100% growth) § 179 developers signed up for Java & CF Meet Ups § Bluemix a major summit feature w/ a successful signup drive § 67 speakers delivered 60 keynotes, panels, talks & sessions § Conference organizers encountered many new dev candidates
A sizing to support 1,000 apps on Cloud Foundry
• 60 virtual machines:
o 20 high memory DEAs
o 11 CF fabric components
o 26 service broker and service nodes
• 160 CPU
• 500 GB memory
• 1.5 TB VM disk
• 200 GB block storage
• 10 floating IPs
+
BOSH deploys and manages Cloud Foundry clusters
Deployment Manifest • Release name/version • # VMs, Job params • Stemcells to use
Stemcell • Base OS • BOSH Agent
Release • Name
Jobs • Software Packages • Config Templates • Scripts
Deployed Environment
irtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software Packages
Virtual Machine • Configuration • Software Packages
Stemcells In a cloud platform, VMs are usually cloned from a template. A stemcell is a VM template containing a standard Ubuntu distribution. A BOSH agent is also embedded in the template so that BOSH can take control of VMs cloned from the stemcell.
Jobs A job is a collection of software which serves a particular purpose (e.g. MySQL or the Cloud Controller). At deployment time, each job will be install on its own stemcell VM.
Releases A release contains a number of jobs which can be deployed into the target environment. A deployment can consist of more than one release and not every job in a release must be deployed.
The BOSH Cloud Provider Interface (CPI) manages the IaaS resources that Cloud Foundry needs
Stemcells create_stemcell(image, cloud_properties) delete_stemcell(stemcell)
Virtual Machines
create_vm(agent_id, stemcell, resource_pool, networks, disk_locality, env) delete_vm(vm) reboot_vm(vm) configure_networks(vm, networks)
Disks
create_disk(size, vm_locality) delete_disk(disk) attach_disk(vm, disk) detach_disk(vm, disk)
Configure OpenStack for Cloud Foundry § Pool of static / floating IP addresses!
§ Support for persistent disks using Cinder backed by SAN!
§ VMs require outbound Internet connectivity!
§ Increased tenant quota for instances needed by Cloud Foundry (~50)!
§ Public/private key pairs !
§ Custom flavors (VM configurations) created according the CF specs!
§ Appropriate security groups for network ports!
Scaling the OpenStack configuration Optimize internal communication!
• Configure OpenStack for scaled concurrency!!
Optimize performance!• Configure OpenStack scheduler to evenly distribute load!
Setup highly available architecture for PaaS workloads!
Load Balancer Nodes
Data Nodes
3x
Compute Nodes
15x Storage Nodes
2x
Controller Nodes
2x 6x
Scaling the Cloud Foundry configuration Optimize internal communication!
• Configure messaging bus for VM communication!!
Optimized routing and bandwidth allocation!• Isolate Cloud Foundry components using multiple networks!
!Maintain Cloud Foundry’s highly available architecture!
Service Gateways
10x Routers 3x
Service Nodes
15x
DEAs 20x
Cloud Controllers
2x
Go try them out!
OpenStack § Join the community, see the docs
http://openstack.org http://docs.openstack.org
§ Get started
http://www.openstack.org/software/start/ § DevStack for local evaluation
http://devstack.org
§ Optimizing OpenStack and Cloud Foundry for large deployments - My team’s talk from OpenStack Atlanta https://ibm.biz/BdFHAH
Cloud Foundry § Join the community, see the docs
http://cloudfoundry.org http://docs.cloudfoundry.org
§ Browse the source http://github.com/cloudfoundry
§ BOSH-lite for local evaluation https://github.com/cloudfoundry/bosh-lite
§ OpenStack and Cloud Foundry, a match
made in heaven - My team’s talk from CF Summit https://ibm.biz/BdFHAr
OpenStack and Cloud Foundry!Pair the leading open source Infrastructure-as-a-Service!with the leading open source Platform-as-a-Service! Daniel Krook!Senior Certified IT Specialist, IBM!@danielkrook – krook.info