Dive Into OpenStack Jesse Andrews & Vish Ishaya Anso Labs
May 12, 2015
Dive Into OpenStackJesse Andrews & Vish Ishaya
Anso Labs
Overview of OpenStackBackgroundCompute: NovaObject Storage: Swift
A Story of Three DeploysKicking the TiresDigging InGetting Physical
Internals / Q & A
The Mission
"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public
and private cloud providers regardless of size, by being simple to
implement and massively scalable."
and many more…
+&
creating open source software to buildpublic and private clouds
Software to provision virtual machines on standard hardware at
massive scale
Software to reliably store
billions of objects
distributed across standard hardware
History
History
Swift Components The Ring: Maps names to entities (accounts, containers, objects) on disk.
Stores data based on zones, devices, partitions, and replicas Weights can be used to balance the distribution of partitions Used by the Proxy Server for many background processes
Proxy Server: Request routing, exposes the public API
Replication: Keep the system consistent, handle failures
Updaters: Process failed or queued updates
Auditors: Verify integrity of objects, containers, and accounts
Account Server: Handles listing of containers, stores as SQLite DB
Container Server: Handles listing of objects, stores as SQLite DB
Object Server: Blob storage server, metadata kept in xattrs, data in binary format Recommended to run on XFS Object location based on hash of name & timestamp
Using OpenStack
Kicking the TiresVagrant is a tool for building and distributing virtualized environments on your laptop/desktop.
1. Install VirtualBox 4
2. Install ruby, rubygems, vagrant
3. Config uses maverick, chef, debs
$ sudo gem install vagrant --pre$ git clone git://github.com/ansolabs/openstack-cookbooks -b devpackages$ mkdir aptcache; mkdir chef; cd chef$ curl -o Vagrantfile https://gist.github.com/raw/786945/solo.rb$ vagrant up
Demo using vagrant
http://ansolabs.com/deploy
Digging Innova.sh is a simple script for setting up a development environment for nova.
1. Install Maverick (in a VM or physical box)
2. Download & run nova.sh
3. Code, test, commit, repeat!
$ curl -O https://github.com/vishvananda/novascript/raw/master/nova.sh$ chmod 755 nova.sh$ ./nova.sh branch$ ./nova.sh install$ sudo ./nova.sh run
Demo using nova.sh
http://ansolabs.com/deploy
Getting Physical
Hardware SelectionOpenStack is designed to run on industry standard hardware, with flexible configurations
Computex86 Server (Hardware Virt. recommended)
Storage flexible (Local, SAN, NAS)
Object Storagex86 Server (other architectures possible)
Do not deploy with RAID (can use controller for cache)
Server Vendor SupportFind out how much configuration your hardware
vendor can provideBasic needs
BIOS settings (hardware virtualization)Network boot IP on IPMI card
Advanced supportHost OS installation
Still get management network IP via DHCP
Network Device ConfigurationBuild in a manner that requires minimal change
Lay out addressing in a block based model Go to L3 from the top of rack uplink
Keep configuration simpleMore bandwidth is better than advanced QoS Let the compute host machines create logical
zones
Host NetworkingDHCP for the management network
Infinite leasesBase DNS on IP
pod-a-10-241-61-8.example.orgOpenStack Compute handles IP provisioning for all
guest instancesCloud deployment tools only need to setup
management IPs
Host OS Seed InstallationBOOTP / TFTP – Simple to configure
Security must be handled outside of TFTPHost node must be able to reach management
system via broadcast request Top of rack router can be configured to forward
GPXENot all hardware supports Better concurrent install capability than TFTP
Host OS InstallationBuilding a configuration based on a scripted
installation is better than a monolithic “golden image”Preseed for Ubuntu / Debian hosts Kickstart for Fedora / CentOS / RHEL hosts YaST for SUSE / SLES hosts Remote bootstrapping for XenServer / Hyper-V
hosts
Scripted configuration allows for incremental updates with less effort
Post OS ConfigurationUtilize a configuration management solution
Puppet: lp:nova contrib/puppet Chef: http://github.com/ansolabs/openstack-cookbooks Cfengine: patches welcome
Create roles to scale out controller infrastructure Support: Queue, Database, LDAP Interfaces: API controllers Workers: Compute, Storage, Networking
Automate registration of new host machines Base the configuration to run on management net IP
Maintain your cloud using technologies you already use to maintain your datacenter(s)Nagios – alertingGanglia – performanceSyslog – log aggregation
Splunk / Ossec
Monitoring
Coming SoonIPv6 support
Live migration
Puppet deployment tools
Snapshotting
Ceph, SheepDog block storage support
Web UI
Network refactor to use open vswitch
Thanks!
Questions? Answers?
Get started at http://ansolabs.com/deploy
Some content “stolen” from
http://www.slideshare.net/bpiatt/openstack-tutorial
LDAP Data Store
Ubuntu OS
Puppet
PXE
Cloud Node
RabbitMQ
SQL DB / KVS
Scheduler
HTTP API
Ubuntu OS
Puppet
KVM
LibVirt
802.1(q)
Brctl
PXE
Project VLAN
Running Instance
Compute Node
Ubuntu OS
Puppet
LVM
iSCSI
PXE
Exported Volume
Volume Node
Ubuntu OS
Puppet802.1(q
)
Brctl
PXE
Project VLAN
Network Node
IPTables
Public Internet