Mesos, DC/OS and the Architecture of the New Datacenter

© 2016 Mesosphere, Inc. All Rights Reserved.

MESOS, DC/OS AND THE ARCHITECTURE OF THE NEW DATACENTER

1


WHOAMI

2

[email protected] databasesPhD, High energy nuclear physics2007 - 2014

2014 - 20152015 -

mailto:[email protected]?subject=

3

PHYSICAL (x86) VIRTUAL HYPERSCALEMAINFRAME

SERVER VIRTUAL MACHINE

PARTITION (LPAR)

UNIT OF INTERACTION

● ERP, CRM, PRODUCTIVITY, MAIL & WEB SERVER

● LINUX, WINDOWS

● DATA / TRANSACTION PROCESSING

● UNIX, IBM OS/360DEFINITIVE

APPS AND OS

● ERP, CRM, PRODUCTIVITY, MAIL & WEB SERVER

● HYPERVISOR + GUEST OS

● BIG DATA, INTERNET OF THINGS, MOBILE APPS

● ???

???DATACENTER

NEW FORM FACTOR FOR DEVELOPING AND

RUNNING APPS

● BIG DATA, INTERNET OF THINGS, MOBILE APPS

● DATACENTER OPERATING SYSTEM (DC/OS)

EVOLUTION OF APPLICATIONS


STATIC PARTITIONING

4

Deals well with heterogeneous workloads.

Headroom left for peak workloads will lead to poor overall resource

utilization.When resource requirements change

potentially costly reconfigurations.


SHARED RESOURCES

5

Multiple frameworks can use the same cluster resources, with

their share adjusting dynamically.

© Gerard Julien/AFP

Run everything in containers!


What about container management?

7


What about container management?

8


MESOS

9


THE BIRTH OF MESOS

10

TWITTER TECH TALK

The grad students working on Mesos give a tech talk at Twitter.

March 2010

APACHE INCUBATION

Mesos enters the Apache Incubator.

Spring 2009

CS262B

Ben Hindman, Andy Konwinski and Matei Zaharia create “Nexus” as

their CS262B class project.

MESOS PUBLISHED

Mesos: A Platform for Fine-Grained Resource Sharing in the Data

Center is published as a technical report.

September 2010

December 2010

DC/OS

April 2016


VISION

11

Sharing resources between batch processing frameworks• Hadoop• MPI• Spark

What does an operating system provide?• Resource management• Programming abstractions• Security• Monitoring, debugging, logging


• A top-level Apache project• A cluster resource negotiator• Scalable to 10,000s of nodes• Fault-tolerant, battle-tested• An SDK for distributed apps• Native Docker support

12

Apache Mesos


ARCHITECTURE

13

Mesos Fundamentals

Now called Agent


TWO-LEVEL SCHEDULING

14

Scheduler(s) AllocationUser

Wor

Resource

Launch


TWO-LEVEL SCHEDULING

15

© 2016 Mesosphere, Inc. All Rights Reserved. 16

YARN


CONTAINER NETWORKING

17



18

Containers isolate tasks on the agent, but what about their communication?

The status quo in a Mesos cluster: one IP per agent.

Many containers per agent: they must share a single IP.

Agent

ContainerContainer

ContainerContainer

ContainerContainer



19

This causes headaches:Port conflictsSecurity compromisesPerformanceService discovery

Agent

ContainerContainer

ContainerContainer

Web serviceWeb service



20


Agent

ContainerContainer

ContainerContainer

Test serviceProd. service



21


Agent

ContainerContainer

ContainerContainer

ContainerContainer



22


Agent

Container Container

Agent

Container Container


IP PER CONTAINER

23

Segregating containers’ network traffic can solve these problems in an elegant,

maintainable way.


CALICO NETWORK ISOLATION

24

Agent

iptables

Kernel routing

Container

Calicomodul


CALICO NETWORK ISOLATION

25


DISTRIBUTED SYSTEMS SDK

26

“Developers will ask for an API not for a VM to

run their software”


Distributed System SDK

27

Focus on application logic, not on data-center structure• Avoid networking-related code• Reuse of built-in fault-tolerance and high availability• Reuse distributed (infrastructure) frameworks (e.g.,

storage)

Mesos


FIRST FRAMEWORK/SERVICE

28

Mesos

Utilize resources ResourceOffer

React on Tasks EventsTaskStatus Update

Scheduler Interface


RENDLER

29

Mesos

Scheduler Interface

Examples implementations for different languages:C++, Go, Haskell, Java, Python, Scala

https://github.com/mesosphere/RENDLER


Datacenter Operating System (DC/OS)

Distributed Systems Kernel (Mesos)

Big Data + Analytics EnginesMicroservices (in containers)

Streaming

Batch

Machine Learning

Analytics

Functions & Logic

Search

Time Series

SQL / NoSQL

Databases

Modern App Components

Distributed systems kernel to abstract resources

Ecosystem of frameworks & apps

Consistent architecture to run on top of kernel

User Interface (GUI & CLI)

Core system services (e.g., distributed init, cron, service discovery, package mgt & installer, storage)

Any Infrastructure (Physical, Virtual, Cloud)31

DC/OS ENABLES MODERN DISTRIBUTED APPS


DC/OS is … ‣ 100% open source (ASL2.0)

• A big, diverse community‣ An umbrella for ~30 OSS projects

• Roadmap and designs• The build tool chain• Docs and tutorials

‣ Not limited in any way‣ Familiar, with a few new features


http://dcos.io

33

http://dcos.io


QUESTIONS?

34

http://apache.mesos.orghttp://dcos.iohttp://mesosphere.com

http://apache.mesos.org

http://dcos.io

http://mesosphere.com

Mesos, DC/OS and the Architecture of the New Datacenter

Data & Analytics