AWS Repatriation: Bring Your Apps Back
Post on 20-May-2015
4144 Views
Preview:
DESCRIPTION
Transcript
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
AWS RepatriationBring Your Apps Back
OpenStack Summit, Hong KongNovember 5th, 2013
@randybias
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
AWS Repatriation
Pub Cloud RepatriationBring Your Apps Back
OpenStack Summit, Hong KongNovember 5th, 2013
@randybias
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
AWS RepatriationPublic Cloud Repatriation
Apps for Hybrid CloudWhat Hybrid-ized Apps Need from Hybrid Cloud
OpenStack Summit, Hong KongNovember 5th, 2013
@randybias
OpenStack Foundation Board of Directors
Prod. OpenStack pioneer, Cloudscaling:Part of OpenStack community since July 2010 (launch)Wins: KT, Internap, LivingSocial, Seagate & others
Advisor to dotCloud (docker.io)
Top 10 Cloud Computing Pioneer
Who
5
I run an OpenStack product company... opinions expressed here are mine
Not the foundation, not the community, just one POV
I believe the pioneers to emulate are:
I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switches
My Bias
6
Live broadcast from OpenStack Summit
Nov 6th, 10AM Hong Kong / 6PM PST
Register here:
http://cloudscaling.com/stack
FYI: State of the Stack v2
7
8
1 Why repatriate from public clouds?
3 Repatriation & behavorial parity
4 Understanding systems contracts
5 Compatibility vs interoperability
6 OpenStack is not a system
7 OpenStack is key to a hybrid cloud future
2 Repatriation requirements
8 Wrapping up
Public & Private Cloud Strengths
11
PublicCloud
PrivateCloud
Large Resource Pool
General Purpose
OptimizedPurpose
RenterEconomics
OwnerEconomics
Direct Control
Public & Private Cloud Drawbacks
12
• Expensive at scale• Limited control for optimization• 3rd party dependency & trust
• Higher upfront capital outlay• IaaS architectural expertise needed• IaaS operational expertise needed
PublicCloud
Large Resource Pool
PrivateCloud
General Purpose
OptimizedPurpose
RenterEconomics
OwnerEconomics
Direct Control
Cost - Own the Base, Rent the Spike
14
Days
Workload Demand
1 2 3 4 5 6 7
Utilized Capacity
Rent the Spike
Private Cloud
Public Cloud
HybridCloud
Own the Base
Control - Optimize Hardware for Workloads
15
CPU & IO Constrained
CPU
RAM
DISK
Network IO
CPU
RAM
DISK
Network IO
Memory Constrained
$$$$
$$$
$$
$$$$
$$$
Compliance - and Security
16
Public clouds will close security gapAlready made exceptional progress
Many public clouds are more secure than the average enterprise
Some security requirements may be impossibleSpecific hardware difficult for public clouds to implement as a solution
Many won’t use public for key workloadsRegional differences in regulatory compliance
Hybrid Cloud Enables Repatriation
17
PublicCloud
PrivateCloud
Manage apps on public and private cloud as a single entity
• Unpredictable Workloads • Scalable Data Processing • Cloudbursting
• Predictable Workloads • Sensitive Data Storage• Reverse Cloudbursting
Agility
Choice
Flexibility
HybridCloud
+
Large Resource Pool
General Purpose
OptimizedPurpose
RenterEconomics
OwnerEconomics
Direct Control
Requires Compatibility & InteroperabilityBuild private cloud to similar spec as the top public clouds if you want:
Economies of scale: Operational Costs
Economies of scale: HW pricing parity
Application portability
Elastic cloud architecturescattle vs. pets
Repatriation Isn’t Automatic
19
1
2
3
4
Example: OpenStack on a Vblock
21
AWS doesn’t run on a Vblock and never willNeither does GCE, RAX, Azure, or any top public cloud
3-5x more expensive than AWS retail priceUpwards of 8x more expensive than wholesale
OpenStack on a Vblock for non-cloud appsSAP, Oracle 10g, vSphere support, etc.
This is the Virtualization 2.0 approachSo-called “software-defined datacenter”
Example: OpenStack via OCS
22
50% savings over AWS over 3-5 years
Time
Public Cloud@ 20% Annual Growth($17.4m over 5 Years)
Yr1 Yr2 Yr3 Yr4 Yr5
$1m
$2m
Cost / Yr
$3m
$4m
Private Cloud@ 20% Annual Growth($9.6m over 5 Years)
2.3
2.8
3.4
4.0
4.8
2.7
1.41.6
1.82.2
Example: OpenStack via OCS
23
OCSw/ Support
AWS 1 Year Resw/o Support
AWS On Demandw/o Support
m1.small $13 $24 $44
m1.medium $25 $49 $88
m1.large $50 $97 $175
m1.xlarge $101 $194 $350
50% savings over AWS over 3-5 years
Top (Elastic) Clouds
24
0% 20% 40% 60% 80% 100%
Windows Azure
GCE
RAX
AWS
% of Respondents
Public Cloud Usage
Running apps
Experimenting
Source: RightScale State of the Cloud Report 2013
Hybrid cloud only makes sense with the leaders; all elastic
Make Private Clouds Look Like Major Public Elastic Clouds?
25
Assume application manages its own fateScaling, self-healing, data replication
Commodity hardware, not gold-platedSmart apps don’t need infrastructure resiliency
Operational model: cattle vs. petsIn public clouds, servers die all the time & no one cares
ALL of the top public clouds are elastic
Behavior MattersMoving apps between clouds is hardAbstractions and APIs can only do so much
Impedance mismatches are commonAny two clouds don’t behave the same
Some clouds are very very similarGCE and AWS are 95% semantically, architecturally, and behaviorally equivalent
This is better than what you find with any two OpenStack cloudsYet these are proprietary software stacks
Let’s look at some examples27
Ex 1: VM Spinup TimesCloud #1: spins up a VM in 5 mins or lesse.g. AWS, GCE, RAX, Azure
Cloud #2: spins up a VM in 60 minutese.g. SFR in France
Write an app deployment frameworkIf you build on cloud #1 first
Assume 5 minutes and retry when a VM fails to show upCauses a failure on cloud #2 when I switch
If you build on cloud #2 firstWorks on cloud #1Now “auto-scaling” takes 1 hour response time
28
Ex 2: Block Storage SnapshottingCloud #1: has incremental snapshottinge.g. AWS, Azure
Cloud #2: has only full snapshotting e.g. cinder backup via OpenStack
Write an app deployment frameworkmanages persistent disks as a mechanism for advanced deployment methods (e.g. EngineYard on AWS)
Build on Cloud #1Snapshot & restore times on cloud #2 cause failures
Build on Cloud #2In testing the non-incremental snapshots mean I can’t use block storage as needed
29
Ex 2: Missing or Different Features
Cloud #1: OpenStack w/ floating IP auto-assignment
Cloud #2: OpenStack w/o auto-assignmentWrite an app deployment framework:
manages VM spinup on behalf of applicationBuild on Cloud #1
Once I try to run on cloud #2, it all breaks because I have to add a check for whether floating IP is there and then add if not
Build on Cloud #2Probably runs on cloud #1
30
Repatriation Checkpoint
31
Repatriation often makes sensefor Cost, Control, Security and Compliance
Repatriation isn’t automaticit requires compatibility & interoperability
Cloud behavior matters
How to enable compatibility and interoperability in a hybrid cloud world?
Why Are We Talking About Systems?
33
We need a common understanding of cloud design principles to agree on what makes a hybrid cloud possible.
Definition of a System
34
1) A set of components
2) Integrated into a specific architecture designed for a purpose
3) Greater than the sum of its parts
System Taxonomy
35
API
Behavior
Semantics
Architecture
How to interface with the system
What the system actually does
How the components fit together
Names & definitions that describe the architecture
Input
Output
System Example: Automobile
36
Gas/brake pedals, steering column, climate controls, etc.
Accelerate, decelerate, turn wheels, power lights, heat/cool cabin, component monitoring, deploy air bags, etc.
API
Behavior
Semantics
Architecture
Input
Output
System Example: Private Cloud
37
API
Behavior
Semantics
Architecture
Input
Output
Compute
BlockStorage
ObjectStorage
Networking
RunInstancesAllocateAddressCreateVolumeAttachVolumeCreateSnapshot...
Launch a VMGet a floating IPCreate a volumeAttach a volumeTake a snapshot...
Explicit vs Implicit Contracts
39
Explicit Contract
Implicit Contract
Precisely and clearly expressed or readily observable. Leaves nothing to inference.
Implied though not directly expressed. Inherent in the nature of something.
API
Behavior
Semantics
Architecture
Input
Output
Explicit vs Implicit Contracts
40
Explicit Contract
Implicit Contract
nova boot --flavor 2 --image e0b7734d-2331-42a3-b19e-067adc0da17d \ --block_device_mapping vda=3195a5a7-fd0d-4ac3-b919-7ba6cbe11d46:::0 myInstanceFromVolume
VM available in 4 min95% of time
API
Behavior
Semantics
Architecture
Input
Output
Explicit vs Implicit Contracts
41
Critical for Compatibility
Critical for Interoperability
API
Behavior
When System is a Black Box
Delivering a Service...
Input
Output
Compatibility versus Interoperability
These are NOT the sameCars and semi-trucks (aka “lorries”) are compatible but not interoperable
Two cars of the same kind are interoperable
Compatibility is a consistent interfacethe explicit contract
Interoperability is an equivalent (and interchangeable) behavior
43
Compatibility versus Interoperability
44
All systems are like this, even clouds
GCE
RAX
VMW
AWS & GCE are highly similar (semantically, behaviorally and architecturally)
This architecture is Elastic Cloud
AWS & Rackspace Cloud are mostly similar
e.g. Rackspace control plane spans AZs, but still it’s Elastic Cloud
AWS & VMware clouds (like Savvis) are very dissimilar
e.g. Fixed vs variable VM sizes
AWS
Definition of a System
46
1) A set of components
2) Integrated into a specific architecture designed for a purpose
3) Greater than the sum of its parts
Definition of a System
47
1) A set of components
2) Integrated into a specific architecture designed for a purpose
3) Greater than the sum of its parts
OpenStack is a component framework w/o an integrated reference architecture
CLI toolsDashboard Other tools
ComputeNetworkingOrchestration
Hypervisor(s) QueuingDatabase
/ KVS/ Cache
External Block
Provider
Physical Network Provider
Provisioning
Log Aggregation
Health Monitoring
etc.
REST Meter Data
REST
SQL, etc. Varies Varies Varies AMQP/0MQ
Topology & Metadata
MeteringREST
REST REST
DNS
ImageManagement
IdentityManagement
REST
Block Storage
Object Storage
OpenStack Component Framework
48
UI Layer
ElasticServices
Layer
Oth
er s
tuff,
you
pro
babl
y ne
ed/w
ant
Data &Resource
Layer
SharedServices
Layer
CLI toolsDashboard Other tools
ComputeNetworkingOrchestration
Hypervisor(s) QueuingDatabase
/ KVS/ Cache
External Block
Provider
Physical Network Provider
Provisioning
Log Aggregation
Health Monitoring
etc.
REST Meter Data
REST
SQL, etc. Varies Varies Varies AMQP/0MQ
Topology & Metadata
MeteringREST
REST REST
DNS
ImageManagement
IdentityManagement
REST
Block Storage
Object Storage
OpenStack Component Framework
49
UI Layer
ElasticServices
Layer
Oth
er s
tuff,
you
pro
babl
y ne
ed/w
ant
Data &Resource
Layer
SharedServices
Layer
Which Networkingor Plugin?
Which Hypervisors& Hardware?
Which Block
Storage?
Which Object
Storage?
Which Orchestration?
Where to Store
Images?
Which Identity
Management?
Configure the Metering?
Use the Dashboard?
Which CLI & Mgmt
Tools?
OpenStack Component Framework
50
Very flexible like the Linux kernel
Components don’t have to be used in the same architecture
Metering / Billing Platform
Private / Public Cloud
PaaS Platform
Storage Platform
OpenStackComponents
Cloud App Mgt Framework
Making OpenStack Compat./Interop. with Elastic Public Clouds
52
Some flavors of OpenStack need to embrace the architecture, semanticsand behavior of elastic clouds
This will allow true Hybrid Cloud for:Reduced friction for Cloud Application Management toolsEasy repatriation and expatriation of apps
The API is the track gauge for compatibility. The rest is what you put on the tracks.
AWSGCERAXAzure...
The Track Gauge Wars
53
OpenStack’s inherent flexibility means winning the compatibility and interoperability race
http://schools-wikipedia.org/wp/g/Great_Western_Railway.htm
The Track Gauge Wars
53
OpenStack’s inherent flexibility means winning the compatibility and interoperability race
http://schools-wikipedia.org/wp/g/Great_Western_Railway.htm
Cloud is Somewhere Around Here
OpenStack is the Key to that Future
55
OpenStack’s perceived weakness is its greatest strength
Flexibility & NOT being a pre-integrated system means it can take many forms
Some of these can be public cloud compatible (OpenStack’s future dominance depends on it)
Public Cloud Compatibility and Interoperability requires...
56
Same APIs (the explicit contract)Same Behavior (the implicit contract)
API and Behavioral Equivalency requires Same Semantics Same Architecture
Hybrid Cloud Brings Both Worlds Together
57
PublicCloud
Scalable
PrivateCloud
General Purpose
OptimizedPurposeOn Demand
RenterEconomics
OwnerEconomics
Secure
Controlled
Manage apps on public and private cloud as a single entity
Hybrid Cloud
Bridging the great divide!
The Hybrid-First Cloud Strategy
58
1) Embrace public cloud architecturesAWS, GCE, RAX, Azure, and possibly vCloud
2) Expand work on refstack & TempestPublic cloud interoperability & hybrid cloud
3) Embrace & extend existing AWS testseutester, aws-compat, OpenStack tempest, NetflixOSS
Public Cloud App RepatriationPrivate Cloud Can Be Cost EffectiveAs long as it’s designed like a public cloud
Hybrid Cloud Provides Best of BothYou probably want maximum choice
Hybrid Requires Behavioral EquivalencyInteroperability and Compatibility are Real
Hybrid-first Cloud Strategy...and OpenStack is the key to building many different hybrid cloud solutions
60
1
2
3
4
Q & A
http://simplicityscales.com/engineering blog
61
Randy BiasCEO & Co-founder, CloudscalingDirector, OpenStack Foundation@randybias
top related