Mirantis Folsom Meetup Intro

OpenStack Overview

Mirantis, Inc.

What is OpenStack?

"Open source software for building private and public clouds"

OpenStack capabilities

● VMs on demand○ provisioning○ snapshotting

● Volumes● Networks● Multi-tenancy

○ quotas for different users○ user can be associated with multiple tenants

● Object storage for VM images and arbitrary files

OpenStack History

● July 2010 - Initial announcement● October 2010 - Austin Release● February 2011 - Bexar Release● April 2011 - Cactus Release● October 2011 - Diablo Release● April 2012 - Essex Release● October 2012 - Folsom Release● April 2013 - Grizzly Release

OpenStack Projects

● Nova (Compute)● Glance (Image Service)● Swift (Object Store)● Keystone (auth)● Horizon (Dashboard)● Quantum (Network) formerly nova-network

● Cinder (Block Storage) formerly nova-volume

OpenStack Projects: Relationship

OpenStack: Deployment Topologycontrol cluster

Horizon Keystone

GlanceQuantum

CinderCloud Ctrl.(nova)

compute cluster

computenode

computenode

computenode

computenode

computenode

computenode

storage cluster

storagenode

storagenode

storagenode

storagenode

storagenode

storagenode

"one to one" relationship with each deployment instance.

nova: Compute

Swift

nova: Cloud Controller

OpenStack Projects: Detailed View (Essex)

nova-api

scheduler

nova-network

nova-volume

nova-db

queue

nova-compute

hypervisor

Glance

glance-api

object store

glance-registry

Keystone

Shared Storage

endpoint

proxy-server

Horizon CLI

nova: Compute

Swift

nova: Cloud Controller

Still part of nova

nova-api

scheduler

nova-network

nova-volume

nova-db

queue

nova-compute

hypervisor

Glance

glance-api

object store

glance-registry

Keystone

Shared Storage

endpoint

proxy-server

Horizon CLI

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin

quantumserver

Cinder

endpoint

cinder-db

scheduler

queue

cinder-vol

controller

nova-api

scheduler

Keystone

keystoneserver

keystone-db

Glance

glance-api

glance-registry

glancedb

block storage

node

storage

compute node

Hypervisor

Network

VM

nova: Computenova-

compute

OpenStack Projects: Communication TypesHTTPAMQPSQLnatv. APIiSCSI

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Use Case: provisioning a VM

Initial State Tenant is created, provisioning quota is available, user has an access to Horizon/CLI

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

User specifies VM params: name, flavor, keys, etc. and hits "Create" button

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

Step 1: Request VM Provisioning via UI/CLI

block storage node

storage

network node

dhcp serv.

router/gw

Step 1: Request Provisioning via UI/CLI

● From UIa. Login to Horizonb. Specify params of VM in "Create VM" formc. Hit "Create" button

● Under the hooda. Form params are converted to POST datab. "Create" request initiate HTTP POST request to back-

end■ To Keystone if auth token is not cached■ To nova-api if auth token hasn't expired yet

Horizon

"The OpenStack Dashboard (Horizon) provides a baseline user interface for

managing OpenStack services."

Step 2: Validate Auth Data Horizon sends HTTP request to Keystone. Auth info is specified in HTTP headers.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Keystone

"Keystone is an OpenStack project that provides Identity, Token, Catalog and Policy services for use specifically by

projects in the OpenStack family."

Keystone Architecture

Keystone: auth flow

Step 2: Validate Auth DataKeystone sends temporary token back to Horizon via HTTP.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 3: Send API request to nova-apiHorizon sends POST request to nova-api (signed with given token).

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 4: Validate API Token nova-api sends HTTP request to validate API token to Keystone.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 4: Validate API Token Keystone validates API token and sends HTTP response with token acceptance/rejection info.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 5: Process API request nova-api parses request and validates it by fetching data from nova-db. If the request is valid, it saves initia db entry about VM to the database.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

nova-api

"nova-api is a RESTful API web service which is used to interact with nova"

nova-api clients

nova database

"nova database stores current state of all objects in compute cluster."

Step 6: Publish provisioning request to queue nova-api makes rpc.call to scheduler. It publishes a short message to scheduler queue with VM info.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 6: Pick up provisioning request scheduler picks up the message from MQ.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Message queue

"Message queue is a unified way for collaboration between nova

components."

Step 7: Schedule provisioning Scheduler fetches information about the whole cluster from database and based on this info selects the most applicable compute host.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

nova-scheduler

"nova-scheduler is a daemon, which determines, on which compute host

the request should run."

nova-scheduler: available schedulers

Scheduler Description

Chance Picks a host that is up at random

Simple Picks a host that is up and has the fewest running instances

Filter Picks the best-suited host which satisfies selected filter

Multi A scheduler that holds multiple sub-schedulers

Step 8: Start VM provisioning on compute nodeScheduler publishes message to the compute queue (based on host ID) and triggers VM provisioning

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 9: Start VM rendering via hypervisornova-compute fetches information about VM from DB, creates a command to hypervisor and delegates VM rendering to hypervisor.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

nova-compute

"nova-compute is a worker daemon, which primarily creates and terminates

VMs via hypervisor API."

nova-compute drivers

nova-compute

XCP

VM

VM

VMWare

VM

VM

HyperV

VM

VM

LPAR

VM

VM

libvirt

KVM

VM

VM

Xen

VM

VM

Qemu

VM

VM

LXC

VM

VM

Step 10: Configure networknova-compute makes a call to quantum-api to provision network for the instance

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 11: configure networkquantum configures IP, gateway, L2 connectivity

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 12: Request volume attachmentIt is assumed a volume is already created. Nova-compute contacts cinder to get volume data.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

block storage

nodestorage

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

Step 12: Request volume attachmentNova-compute sets up iSCSI initiator & instructs the hypervisor to mount iSCSI vol. as a new block device

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

block storage

nodestorage

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

Step 12: Request volume attachmentiSCSI communication begins

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

block storage node

storage

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

network node

dhcp serv.

router/gw

cinder

"block storage as-a-service"

cinder architecture

Step 13: Request VM Image from Glancenova-compute requests VM image from Glance via Image ID

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Glance

"The Glance project provides services for discovering, registering, and retrieving virtual

machine images."

Step 14: Get Image URI from GlanceIf image with given image ID can be found - return URI

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Step 15: Download image from Swiftnova-compute downloads image using URI, given by Glance, from Glance's back-end. After downloading - it renders it.

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

Initial State

nova

nova-db

queue

Swift

object store

proxy-server

UI: horizon or CLI

Quantum

quantum-db

quantumplugin/agt

quantumserver

Cindercinder-api

cinder-db

scheduler

queue

cinder-vol

controllernova-api

scheduler

Keystonekeystone

serverkeyston

e-db

Glanceglance-api

glance-registry

glancedb

compute nodeHypervisor

Network

VM

nova: Computenova-

compute

block storage node

storage

network node

dhcp serv.

router/gw

www.mirantis.com