Ubuntu Cloud - EGI (Indico)

Ubuntu Cloud

Deploying Clouds with Ubuntu

Nick Barcet <[email protected]>Cloud Solutions Lead

● Who we are

– Commercial sponsor of the Ubuntu project

– 400+ employees, 30 countries

– 5 offices worldwideLondon, Montreal, Boston, Taipei, Shanghai

– 80% distributed workforce

● Services we provide

– Professional services

– Landscape system management

– Launchpad

– Training, Online services

Canonical

A bit of History

● Oct 2004:

● First release of Ubuntu (Dekstop only)

● Oct 2006:

● First release of Ubuntu Server Edition

● April 2008:

● First distribution to officially support KVM virtualization

● April 2009:

● First official Ubuntu cloud guest images

● First distribution to offer built in IaaS components (Eucalyptus)

● April 2010:

● First definition of Cloud-Init

● First distribution to provide OpenNebula

● Oct 2010:

● First distribution to provide OpenStack

Server Adoption Stats

What is your primary operating system for Linux development?

Ubuntu - 54%

Cloud Market, AMI usage by Platform

What is your primary operating system for software deployment?

Source : 2009 Eclipse Community Open Source Developer Report (Sample size 570)

Ubuntu - 29%

Google Trends

Cloud?

Hardware

Virtualization

Infrastructure

Platform

Application

Cloud computing stack

Storage Network

etc..LenovoDellIBMHP

etc..HyperVVMWareKVMXen

Amazon, GoGrid, 3Tera, Eucalyptus, etc...

GoogleApps, Java, Azure, etc...

Salesforce.com, GoogleDocs, Office, etc...

Our current cloud offering

x86

KVM

Ubuntu Enterprise Cloud InfrastructureStorage Network

Hardware

Virtualization

Infrastructure

Platform

Application

Why?

Speed of Innovation

● Cloud reduces development friction and costs

● Cloud adoption is being driven by developers

● Opportunity

– Access to latest technologies needed for competitive advantage rather than cost saving

– Reduces time to market and therefore increases speed of innovation

Second Wave of OSS Adoption

● First wave in 2000 focused on Linux and LAMP

– Winners were RedHat, MySQL

● Cloud represents Second Wave

– OSS innovating rather than imitating

– OSS is dominant in the cloud

● Opportunity

– Scalability of cloud requires free bits

Ubuntu lifecycle

12

A fundamental change for developers

IaaS is not virtualization

● Virtualisation:

● Is not controllable by the application

● Each Instance require a specific disk image

→ the term “virtual machine” regroups the notion of disk image and execution instance.

● Assumes old separation of work model between developer and operators

Not just a porting exercise

● Current applications do not need to be ported

● anything that will run on a server can run in the cloud

● Application need to take advantage of the cloud

● Dynamic scaling

● Introspection

● Storage

Old Habits

Application

Hardware

Network

Storage

● Assume that the infra is resilient

● Assume that the hardware scales vertically

● Assume that SQL is the only DB method, and that it will grow with the app

● SPOF and scalability are a data center issue

Dev

Re

alm

Op

era

t or

Rea

l m

New Habits

Application

Hardware

Network

Storage

De

velo

pe

r R

ea l

m

● Applications can scale horizontally

● Application can spawns additional instances based on needs

● Any instance can fail, but you can always launch another one

● Data can scale horizontally too if you use NoSQL DBs for critical elements

● Applications can run across data centers (availability zones)

● Application control their storage

APIs your application can use

● Amazon's API to control AWS leads the way to control

● The infrastructure (EC2)

● Elemental/Object storage (S3)

● Block storage (EBS)

● Instances have access to meta-data and user-data

● Can query about environment

● Can learn about self

→ Developers are now operators of the data center: DevOps

IaaS instances particularities

● Instances are very similar to Object instances in programming

● A single Image can have multiple instances

● Can initialize themselves based on user-data passed to it

● Specialization of the instance is not linked to the image

● Storage is not linked to the image

Instance, Image and Storage Example

1. Instance is created from a generic OS image with user data:

• Type: web front end• Load balancer: lb.somewhere.com• EBS: use disk “web”

2. Instance boots, receives and parse user data

3. Installs apache and configures itself

4. Attaches to ebs disk “web”

5. Informs load balancer that it is ready to receive requests

Developing for the Cloud

● Requires to develop ON a cloud infrastructure● Requires an instance OS that is compatible with the

cloud

21

Using Ubuntu IN the cloud

Ubuntu is free, maintained AND supported

● No additional fee to run your instance in public cloud

● Images are maintained, updates are public

● Mirrors are maintained within the environment

● Services, including support, are available as an option

Ubuntu innovates

● Small, efficient base image

● Provides Amazon's API for most languages

● Same image can be ran on multiple cloud

● Amazon

● Ubuntu Enterprise Cloud

● Rackspace*

● Flexiant*

● GoGrid*

● KVM virtualisation environment

● LXC containers*

● Has cloud-init built in

* Coming soon

Cloud-Init

● Generic way to initialize your instance

● Works identically on multiple clouds

● Supports:

● Mounting EBS volumes

● Configuration management tools (ie Puppet)

● Modular and external scripting

● Package installation and configuration

Ubuntu Images – The #1 Public Cloud OS

● Amazon Web Services

– Images available since 2007

– Believed to be the most widely deployed OS– https://help.ubuntu.com/community/EC2StartersGuide

● Rackspace Cloud

– Images available since 2008

– Confirmed as most widely deployed OS– http://cloudservers.rackspacecloud.com/index.php/Ubuntu_-_Setup

26

Using Ubuntu AS the cloud

Ubuntu Enterprise Cloud

Rapid deploymentOptimize resources & immediacy (self service IT)Best of breed (KVM, Eucalyptus) Compatible technology (matches EC2/S3)Supports multiple guest O/S Secure, trusted & open source

Ubuntu Enterprise Cloud

Compatible technology

Ability to use the same Ubuntu machine images and management tools across both private and public systems, minimising costly re-training or application change when moving from private to public and vice versa.

Hybrid Focus

Ubuntu on EC2 (public)Ubuntu Enterprise Cloud (private)Maximise benefits whilst minimising risksElasticitySimplifies burstingCommon StandardsCommon Ubuntu machine image

UEC/Amazon API Compatibility

Ubuntu Enterprise Cloud

Amazon Web Services

Ubuntu Developer Day – India 2011

SLIDE 29 of 41

Source: Ubuntu Server Edition User Survey by Canonical

• Users convinced of the validity of Ubuntu & UEC as a platform for cloud.

• 7% have built a private cloud with UEC, 17% are expecting to in the next 12 months.

• Currently tracking over 20,000 deployments of UEC

UEC Adoption

Five Characteristics:

- On Demand Service- Ubiquitous Network Access- Location Independent Resource Polling- Rapid Elasticity- Measured Service

Delivery Models

- Software as a Service (SaaS)- Platform as a Service (PaaS)- Infrastructure as a Service (IaaS)

Deployment Models

- Private Cloud- Public Cloud- Hybrid Cloud

Ubuntu Enterprise Cloud

Example Use Case

Research Labs

● Characteristics

● Research and academic institutions have multiple labs

● Each lab needs to have its own environment since each lab can have very different configuration

● Pain Points

● Waste of infrastructure resources

● Financial drain on the organization

● Consequences

● Separate IT infrastructure for every lab

● Since each environment is not utilized at all times, a great deal of equipment stands idle when not utilized

● How we help

● Each lab can spin its own environment when needed and spin down when testing is done, while maintaining the environment it needs

● Equipment utilization is maximized

● Money is saved

HPC Cloud sizing

● L = average number of instances/lab (single cores)

● X = Number of labs

Current cost

● L = 2000

● X = 20 (L x X) = 40000 cores→

Projected cost with UEC

● M = 55% (average concurrent usage)

● V = 20% (virtualization overhead) (1.2 x L x N x M) + L) / X / Y = 888 cores, → 30% potential savings

A peak at the future

Roadmap (Confidential)

● 11.04 (Natty)

● OpenStack as a technology preview

● 11.10 (Oneiric)

● OpenStack in main● Bare metal deployment for both Hadoop and OpenStack● Ensemble service orchestration as a tech preview

● 12.04 LTS

● Ensemble in main

Hadoop on Ubuntu

● Cloudera Distribution for Hadoop

● Long term relationship with Cloudera

● Numerous deployment on Ubuntu

● Working on a joint Bare Metal provisioning solution

Thank you!

http://www.ubuntu.com/cloudhttp://cloud.ubuntu.com ← blog aggregatorhttp://help.ubuntu.com/community/UEC ← tuto & doc Twitter @ubuntucloud

@nijaba ← me

euca-add-group euca-delete-snapshot euca-disassociate-address euca-add-keypair euca-delete-volume euca-download-bundle euca-allocate-address euca-deregister euca-get-console-output euca-associate-address euca-describe-addresses euca-modify-image-attribute euca-attach-volume euca-describe-availability-zones euca-reboot-instances euca-authorize euca-describe-groups euca-register euca-bundle-image euca-describe-image-attribute euca-release-address euca-bundle-vol euca-describe-images euca-reset-image-attribute euca-confirm-product-instance euca-describe-instances euca-revoke euca-create-snapshot euca-describe-keypairs euca-run-instances euca-create-volume euca-describe-regions euca-terminate-instances euca-delete-bundle euca-describe-snapshots euca-unbundle euca-delete-group euca-describe-volumes euca-upload-bundle euca-delete-keypair euca-detach-volume euca-version

Example commands delivered by the euca2ools package:

Rich Interface