CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses State of the Stack v2 An Update on Why OpenStack is The New Linux. OpenStack Summit, Hong Kong November 7th, 2013 @randybias
This 2nd major State of the Stack address is a complete refresh of the spring 2013 edition, broadcast live on BrightTALK from the OpenStack Summit in Hong Kong.
Randy Bias, CEO and Co-founder of Cloudscaling examines the progress from Grizzly to Havana and delves into new areas like refstack, tripleO, bare metal server provisioning, the move from "projects" to "programs", and public/hybrid cloud compatibility. Check out the updated statistics on project momentum and look more closely at big upgrades in Havana, including OpenStack Orchestrate (Heat), which has the opportunity to change the game for OpenStack in the greater private and hybrid cloud game. We also discuss the "what is 'core'" debate and examine the idea that OpenStack is a kernel, not a complete cloud OS.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
State of the Stack v2An Update on Why OpenStack is The New Linux.
Prod. OpenStack pioneer, Cloudscaling:Part of OpenStack community since July 2010 (launch)Wins: KT, Internap, LivingSocial, Seagate & others
Advisor to dotCloud (docker.io)
Top 10 Cloud Computing Pioneer
Who
3
I run an OpenStack product company... opinions expressed here are mine
Not the foundation, not the community, just one POV
I believe the pioneers to emulate are:
I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switches
My Bias
4
5
1 What is OpenStack?
3 History & Momentum4 OpenStack Governance5 Stackology - OpenStack at 10km6 Stacking it Up - a dive into the projects
10 Summary
2 Why the Success?
8 Who’s using it and how?7 Stack Gaps
9 RightScale - State of the Cloud Report
What is OpenStack?
OpenStack From 10km
7
Networking
OPENSTACK CLOUD OPERATING SYSTEM
Standard Hardware
Compute Storage
Your Applications
OpenStack Dashboard
OpenStack Shared Services
APIs
OpenStack Mission
8
"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to
implement and massively scalable."
Code Community
OpenStack Foundation Mission
9
The OpenStack Foundation is an independent body providing shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.
The ubiquitous cloud computing platform
The Battle is Over (open src)
10
OpenStack CloudStack Eucalyptus OpenNebula
Source: trends.google.com
OpenStack Launch
Battle is Nearly Over (closed src)
11
OpenStack vSphere vCloud
Source: trends.google.com
Linux 2000 vs. Linux 2009
12
Is this OpenStack’s Trajectory?
Operating system family market share
2000 2009
Unix Linux
Linux Unix
Source: Linux Magazinehttp://www.linux-mag.com/id/7749/
OpenStack Foundation & CommunityEsp., all of the companies who realized this could be big
The Big Enterprises for Driving Interest
Maturity of Community SDLCNo other similar project has this SDLC+velocity
16
1
2
3
OpenStack Development Maturation
17
Regular
Releas
e
Cycle
Professional Open
Source Leaders
Advance
d CI &
Testi
ngCentralized Shared
Libraries (Oslo)
SDLC
Regular Release CycleEvery 6 months, we coordinate & integrate:Thousands of patches & commitsAcross hundreds of developersWhile also increasing # of projectsIn last 3 yrs: grew from 2 to 9 “integrated” or “core” projects+4 more new projects in “incubation” as of Havana
No other related project is capable of thisNo other related project has this many capabilities or is this complex
18
Professional Open Source Leaders
Experienced open source companies:RedHat, IBM, HP, Mirantis, etc.
Dev leaders w/ open source histories:Nova - Russell Bryant (Asterisk)Oslo - Mark McLoughlin (KVM, GNOME, Linux kernel, Java)TripleO & CI system - Monty Taylor (MySQL)... etc.
Very different from origins
19
Advanced Continuous IntegrationMassive Effort à Improved QualityParallel test pipelining w/ grouped, related changesets
Gated CommitsAll Code Has to Jump Through GatesTempest Test Framework
Service Level Testing, Regressions, Full Behavorial Tests, etc.
Nova Neutron CinderKeystone Glance OsloHeat Swift Tempest
0
175
350
525
700
7/2012 12/2012 3/2013 7/2013
Tempest Test Count
Total Tests (Unit + Tempest) Tempest Tests in One Year
Centralized Shared LibrariesProject OsloLess interest in shared code and reuse initiallyRun by experienced leader, Mark McLoughlinWhere all common code goes
Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q3http://www.qyjohn.net/?p=3373
31
For the CloudStack projects, influence from Citrix is quite obvious, over 48% of github.com commits come from accounts belonging to citrix.com and cloud.com.
The focus for these diagrams was ease of reading, not accuracy.
See Ken Pepple’s originals or the code if you need truth.
That being said, our team tried really hard for accuracy.
Blame me for any errors.
Architecture Diagrams Legend
56
CLI toolsDashboard Other tools
ComputeNetworkingOrchestration
Hypervisor(s) QueuingDatabase
/ KVS/ Cache
External Block
Provider
Physical Network Provider
Provisioning
Log Aggregation
Health Monitoring
etc.
REST Meter Data
REST
SQL, etc. Varies Varies Varies AMQP/0MQ
Topology & Metadata
MeteringREST
REST REST
DNS
ImageManagement
IdentityManagement
REST
Block Storage
Object Storage
OpenStack (m)Architecture Slide
57
UI Layer
ElasticServices
Layer
Oth
er s
tuff,
you
pro
babl
y ne
ed/w
ant
Data &Resource
Layer
SharedServices
Layer
OpenStack Compute (Nova)
58
Compute Thoughts
59
Nova still runs best w/ KVMDo we need another hypervisor? What’s the biz case?ESX support is a mess
Multiple Availability Zones still not solvedCells are for making one AZ bigger
complex, tight-couplingWe need a clean sharding mechanism for AZes
also what about Cinder and Neutron?
Cinder/Neutron integr. needs examiningMore information needs to be able to be passed back
Compute (Networking) Thoughts
60
nova-network still requiredNeutron has been L2 focused & L3 gap still exists
centralized “single-host” networking is #faildecentralized “multi-host” is more #failnova-conductor security for hypervisor obviatedmetadata-api, nova-network, or Neutron on all HVs?
security implications
reconciling Neutron and nova-network?Neutron needs more L3 capability, but ...
do we always want Neutron in every deployment?
OpenStack Block Storage (Cinder)
61
Block Storage Thoughts
62
Default “nova-volume” func. is too minimalWhen people think Block Storage service they assume:
Persistent, Network-based, & Performant - it isn’t
Cinder scheduler needs info from NovaAssuming you want to do anything interesting
Point of lock-in since default isn’t usefulYou have to place a bet on a block storage solutionThese are expensive, experimental or proprietaryRely on vendor not community to help w/ problems
OpenStack Networking (Neutron)
63
Networking Thoughts
64
Default networking functionality is minimalThe APIs have been L2-centricL3 functionality is same as existed with nova-network
Same architecture, same basic layout, with all of the downsides
Needs a Neutron plugin for full func.Can’t run more than one plugin at a time per functionNot clear how many production deployments there are:
Good news is that this area is hotSo hopefully this is unstuck soon
OpenStack Object Storage (Swift)
65
Object Storage Thoughts
66
Strength: massive concurrency, proven scaleSwift needs additional attentionToo much focus on Nova and Neutron as the “new shiny”Container replication is a bad hackOriginally most mature of the projects; but has lagged
Good news: area has many new playersSeagate, SwiftStack, RedHat, IBM, eNovance, Mirantis, and HP all deeply involved
Major new features coming:Global clusters for multi-site replicationFeatures for prioritized replicationPluggability for support of DFS like GlusterFS (RedHat)
OpenStack Image Mgmt (Glance)
67
Image Mgmt Thoughts
68
Still not clear why this is standalone projectReally a sub-function of Compute
Semi-pluggable (but not really)Uses different backends for image storage
To be really useful it needs more features:P2V, V2V, and other image conversionAbility to slipstream PV drivers into imagesConvert from popular formats: OVF, AMI, etc.
OpenStack Identity (Keystone)
69
Identity Mgmt Thoughts
70
Mixed identity / schizophreniaVerifies identity, authorization, AND service registry
Service registry is one of manyNova, Cinder, et al have their own internal registries
Slows everything downSee LivingSocial presentation from Folsom SummitMitigated with memcache assuming you turn it on
OpenStack needed to reinvent wheel here?Could have just been LDAP with a schema + caching
OpenStack Dashboard (Horizon)
71
Updates
Model
View Controller
User
Manipulates
Sees Uses
Dashboard Thoughts
72
It’s gotten waaay better!Same UI for end-user and admin is bad idea
CloudStack did this and it was a messThe workflows and views are too differentSecurity considerations exist
General lag: many things aren’t accessible e.g. Heat, TripleO, Marconi, Trove, etc.Can be a 6 month cycle time for new feature UI support
Need better docs on extending, w/o harmCustomers, product companies, SPs all want to modifyAllow for customizations, while supporting upgrades, etc.
OpenStack Metering (Ceilometer)
73
Metering Thoughts
74
Metering systems are hardBad or incomplete data for SPs is existentialMetering system should be *very* baked (is 1yr enough?)
No tokenized meter dataInstance hours not enoughHow do you bill for Windows, Oracle, RHEL licenses?Tokens stack: size of instance, OS and app licenses, etc.
Needs to get flow data from edge switchesNetflow and/or Sflow support for physical switches
Data from the vSwitches is not the best source in the real world
Mirantis Stackalytics - All contributors for “core” in Havanahttp://www.stackalytics.com/?release=havana&metric=loc&project_type=core&module=&company=&user_id=