EMC World 2016 - code.13 State of the Container Ecosystem with Persistent Applications
Post on 08-Feb-2017
545 Views
Preview:
Transcript
1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved.
STATE OF THE CONTAINER ECOSYSTEM WITH PERSISTENT APPLICATIONSSTEVE WONGDEVELOPER ADVOCATE –EMC{code}
2© Copyright 2016 EMC Corporation. All rights reserved.
Old: monolithic, big, slow to change, long lived
APPLICATIONS ARE CHANGINGNew: many instances, rapidly implemented, updated, deployed
3© Copyright 2016 EMC Corporation. All rights reserved.
SOLUTION: ENTER THE CONTAINER
4© Copyright 2016 EMC Corporation. All rights reserved.
TODAY’S AGENDATwo rapidly developing areas fields within the container space:1. Persistent
Storage2. Schedulers
5© Copyright 2016 EMC Corporation. All rights reserved.
WHERE DOES STORAGE FIT INTO THIS?… THE SHELL GAME
Containers and this new “cattle” thing magically made my storage management headaches disappear, right?
6© Copyright 2016 EMC Corporation. All rights reserved.
IF YOUR CONTAINER ARE CATTLE,YOU HAVE NOTHING TO LOSE …
Go ahead, use local storage. • The test: I don’t need
persistence beyond the life of the container instance or host
7© Copyright 2016 EMC Corporation. All rights reserved.
I HAVE SERVICES THAT AREN’T CATTLEWHAT IF I RUN THESE THE SAME WAY?
A portion of the container host’s filesystem can be exposed to a container. • Availability Risk
– Migrate container to another host – your storage is gone
– Host goes down – your service goes down• Scale Limitation
– Need more storage than the host has? Sorry…
8© Copyright 2016 EMC Corporation. All rights reserved.
SOLUTION: MOUNT OF EXTERNAL STORAGE
9© Copyright 2016 EMC Corporation. All rights reserved.
HOW DO I MOUNT EXTERNAL STORAGE?THE REX-RAY PROJECT
• Open Source - extensible• Supports many container
platforms, including cloud hosted• Supports many storage platforms• Managed and configurable by
container host, not inside-the-container configuration
10© Copyright 2016 EMC Corporation. All rights reserved.
TODAY’S AGENDATwo rapidly developing areas fields within the container space:1. Persistent Storage2. Schedulers..we’ll come back to storage in a minute
11© Copyright 2016 EMC Corporation. All rights reserved.
GOT CONTAINERS AT SCALE?YOU NEED A SCHEDULER…
12© Copyright 2016 EMC Corporation. All rights reserved.
ASSUME 1000’S OF CONTAINERS
• How do you place an assortment of containers of different sizes and shapes onto your servers efficiently?
13© Copyright 2016 EMC Corporation. All rights reserved.
ASSUME 1000’S OF SERVICES• How do you
find them?• How do they
find each other?
14© Copyright 2016 EMC Corporation. All rights reserved.
ASSUME 1000’S OF SERVICES
15© Copyright 2016 EMC Corporation. All rights reserved.
ASSUME 1000’S OF SERVICES
It’s Microservice Time!
16© Copyright 2016 EMC Corporation. All rights reserved.
WHAT’S A SCHEDULER?• Manages compute nodes
– Takes reports from each backend server, as to whether they are still available, and what their capabilities and capacities are.
• Manages incoming work request queue– Takes user’s, batch, and API job requests– Based on requirements, decides which
backend server(s) can fulfill each request– If multiple backend servers are suitable,
engages optimization algorithms in favor of random placement
on demand
scheduledbatch
17© Copyright 2016 EMC Corporation. All rights reserved.
MANY SCHEDULERS ARE AVAILABLE• Differences
– Management “philosophy”• “Tell me How” vs “Tell me What”
– aka Imperative vs Declarative, Explicit vs Implicit, declare procedure vs declare outcome– Container technology(s) used– Maturity– Supported scale– Support for external storage volume mounts– Degree of “convergence” with a specific application stack
• Aka degree to which scheduler is “opinionated”
18© Copyright 2016 EMC Corporation. All rights reserved.
SCHEDULERS COMPARED Swarm Kubernetes Mesos Cloud Foundry
Management style
procedure outcome outcome outcome
Maturity new new years yearsScale 1,000? 1000 10,000+ 5,000Extensibility limited limited Well defined limitedContainer Engine
Docker DockerRoadmap: CoreOS
Linux+DockerRoadmap:CoreOS
Garden
External storage support
DVD RexRay in host RexRay in host RexRay
Application specific framework availability
Roadmap:Aurora for generic
proposed 20+ applications,Marathon and Aurora for generic
Multiple language platforms
19© Copyright 2016 EMC Corporation. All rights reserved.
KUBERNETES• Started by Google• Primary container integration is with Docker, but
enhancing support for CoreOS in upcoming release• Implicit aka Declarative configuration philosophy
– Declare desired outcome– User defined Key/value labels on objects can be used to
support this
20© Copyright 2016 EMC Corporation. All rights reserved.
KUBERNETES• pods group containers together’
– pods are the smallest management unit– provides “plumbing” to allow containers to interoperate
(micro-service per container) inside the pods• replication controllers handle container lifecycle• labels provide a way to find and query containers
and other objects• services assemble a set of containers performing a
function
21© Copyright 2016 EMC Corporation. All rights reserved.
KUBERNETES
• monitors health of services, with auto-healing• Load balancing built in• External volume mounts supported in POC / release
candidate with additional external storage features in active planning stage
22© Copyright 2016 EMC Corporation. All rights reserved.
SWARM• Started by Docker Inc.• Stays with Docker CLI/API originally developed for
single host, and allows using it across a multiple node cluster
• Explicit aka Imperative configuration philosophy– Declare desired how to achieve outcome
• Compose uses a yaml file to group containers
23© Copyright 2016 EMC Corporation. All rights reserved.
SWARM• Opportunities for enhancement:
– Container communication across host boundaries– External persistent volume management
24© Copyright 2016 EMC Corporation. All rights reserved.
MESOS• Mesos is the scheduler that provides maximal
flexibility• It allows you to run sub-schedulers (aka
Frameworks), many of which are available off-the shelf – or you can implement your own
• It can support Docker containers, but also supports non Docker workloads.
25© Copyright 2016 EMC Corporation. All rights reserved.
MESOS FRAMEWORKS• Marathon
– Supports generic workloads that run in a linux container– Supports Docker containers
• Hadoop• Spark• Kafka• Many more
26© Copyright 2016 EMC Corporation. All rights reserved.
MESOS STORAGE SUPPORT• External volume mounts supported now• Full global external storage resource management,
using an offer/accept model is in active planning stage now
27© Copyright 2016 EMC Corporation. All rights reserved.
CLOUD FOUNDRY• Cloud Foundry delivers management at the
application level • Containers are used “under the covers” but you
aren’t exposed to the details and don’t have to manage them
28© Copyright 2016 EMC Corporation. All rights reserved.
CONTAINER SCHEDULERS AS PAAS STACKS• A variety of PAAS stacks exist that attempt to give
you “everything you need” to create an app on Amazon, or your own hardware, provided you accept the constraints imposed by the platform.
• But marketing people hate to call these constraints. Instead, the PaaS stack is described as opinionated
– Some container schedulers are more opinionated than others. Basically the platform makes technology choices for you, and you don’t get to change these.
29© Copyright 2016 EMC Corporation. All rights reserved.
CLOUD FOUNDRY• Cloud Foundry is far more opinionated than other
container schedulers – and depending on your objective, and staffing capacity, this could be a good thing
30© Copyright 2016 EMC Corporation. All rights reserved.
TODAY’S AGENDATwo rapidly developing areas fields within the container space:1. Persistent Storage2. Schedulers..we’ll come back to storage in a minute
31© Copyright 2016 EMC Corporation. All rights reserved.
A SCHEDULER SHOULD INTERFACE WITH STORAGE AT MULTIPLE LEVELS
Create
Utilize
Retire
Lifecycle management with governance
Mount managementAssignment to applications(tenancy)
32© Copyright 2016 EMC Corporation. All rights reserved.
33© Copyright 2016 EMC Corporation. All rights reserved.
WHAT POLLY STORAGE SCHEDULER DOES…• Manages interfaces to storage providers
– Multiple concurrent providers anticipated• Mixed vendors
– Including public cloud– Takes reports from each provider, and what their
capabilities and capacities are– Uses admin defined key/value labels to enable
• Outcome based placement• Concurrent use of multiple schedulers, with quotas• Pool advertisement, with suballocation
34© Copyright 2016 EMC Corporation. All rights reserved.
STORAGE SCHEDULER = DELIVER SCALE OUTScale out is not just about adding capacity. A Storage Scheduler will let you:• Add, remove, replace storage provider platforms
– in or out of public cloud• Add, remove, replace schedulers• This is about giving you flexibility and
shifting the burden of keeping up to vendors
35© Copyright 2016 EMC Corporation. All rights reserved.
DEMO
• Demo attach of a persistent volume under Kubernetes
• Shutdown container• Re-attach same volume to a Mesos hosted container,
under Linux containerizer instead of Docker
36© Copyright 2016 EMC Corporation. All rights reserved.
Data Persistence in the New Container WorldWednesday 3PM
Joshua BernsteinVP of Technology for ETD
Tobi Knaup CEO & Co-Founder of Mesosphere
Guru Session
37© Copyright 2016 EMC Corporation. All rights reserved.
@EMCcode@cantbewong
emccode.comcommunity.emccode.comcantbewong.com
Come visit us at Booth #1044 or in the vLab
Questions?
38© Copyright 2016 EMC Corporation. All rights reserved. 38© Copyright 2016 EMC Corporation. All rights reserved.
top related