Intro to fog and openstack jp

Introduction to Fog and Openstack

Hunter Nield - 1/3/2013Twitter: @hunternield

Fogとは• 複数のクラウド・サービスを接続する共通インターフェイス

• オープンソース Ruby ライブラリ (MIT License)

• OpenStack への接続が容易

• http://fog.io

• https://github.com/fog/fog

Fogの歴史

• 2010年 Wesley Beary (geemus on Github) により開始

•初期バージョンは Amazon、Rackspace、Slicehostのみ

Fog と OpenStack の歴史

• Diablo バージョンにて開発開始

• Morphlabs が Essex からサポートを始め、Folsom、その後のバージョンと継続する予定。

BasicsFog provides high level interfaces to cloud services

ServicesFog Services map closely to the Openstack Services

ComputeConnects to the Openstack Nova API for managing

Instances (Servers), Security Groups, Floating IPs, etc

http://rubydoc.info/gems/fog/1.9.0/Fog/Compute/OpenStack

IdentityConnects to the Openstack Keystone API for managing Users, Tenants and Authentication

http://rubydoc.info/gems/fog/1.9.0/Fog/Identity/OpenStack

VolumesConnects to the Openstack Nova/Cinder API for managing Block Storage

http://rubydoc.info/gems/fog/1.9.0/Fog/Volume/OpenStack

ImagesConnects to the Openstack Glance API for managing VM Images and Snapshots

http://rubydoc.info/gems/fog/1.9.0/Fog/Images/OpenStack

StorageConnects to the Openstack Swift API for managing Object Storage

http://rubydoc.info/gems/fog/1.9.0/Fog/Storage/OpenStack

NetworkConnects to the Openstack Quantum API for managing Networks

http://rubydoc.info/gems/fog/1.9.0/Fog/Network/OpenStack

InternalsFog follows a simple structure to handle the

management of cloud services

RequestsQuery the cloud service APIs

Mock

• Test data to return example API responses

• Requires no network connection

Mock (Example) class Mock

def list_flavors response = Excon::Response.new response.status = 200 response.body = { 'flavors' => [ { 'name' => '256 server', 'id' => '1', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '512 server', 'id' => '2', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '1GB server', 'id' => '3', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '2GB server', 'id' => '4', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '4GB server', 'id' => '5', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '8GB server', 'id' => '6', 'links' => ['https://itdoesntmatterwhatshere.heh'] }, { 'name' => '15.5GB server', 'id' => '7', 'links' => ['https://itdoesntmatterwhatshere.heh'] } ] } response end

end

Real

• Returns actual data from the API

Real (Example)

class Real

def list_flavors request( :expects => [200, 203], :method => 'GET', :path => 'flavors.json' ) end

end

Models

• Ruby Object representation of a Cloud Service Entity (Instance, Image, etc)

• Uses one or many ‘Request’ methods to retrieve data about an Entity

• Includes convenience methods for easy access

Models (Example) class Server < Fog::Compute::Server

identity :id attribute :instance_name, :aliases => 'OS-EXT-SRV-ATTR:instance_name'

attribute :addresses attribute :flavor attribute :host_id, :aliases => 'hostId' attribute :image attribute :metadata attribute :links attribute :name attribute :progress attribute :accessIPv4 attribute :accessIPv6 attribute :state, :aliases => 'status' attribute :created, :type => :time attribute :updated, :type => :time

def ready? self.state == 'ACTIVE' end

def reboot(type = 'SOFT') requires :id service.reboot_server(id, type) true end end

Collections

• Ruby Object representation of many groups of Cloud Service Entities (Instances, Images, etc)

• Handles querying and iteration of multiple models of a single type

Collections (Example) class Servers < Fog::Collection

model Fog::Compute::OpenStack::Server

def all(filters = filters) self.filters = filters data = service.list_servers_detail(filters).body['servers'] load(data) end

def get(server_id) if server = service.get_server_details(server_id).body['server'] new(server) end rescue Fog::Compute::OpenStack::NotFound nil end

end

Servers

Server Server Server Server

Collection

Models

Other methodsSpecifically for Openstack

• service.unscoped_token

• The reusable token which can be used to retrieve scoped tokens “auth_token” on each tenant.

• service.auth_token

• The current authentication token sent to OpenStack on each request as a header (“X-Auth-Token”).

• service.current_user

• The Hash representation of the JSON returned information about the current user returned by OpenStack upon authentication.

• service.current_tenant

• The Hash representation of the JSON returned information about the current tenant returned by OpenStack upon authentication.

• service.credentials

• A convenience method to extract needed credentials to easily instantiate a new usable service object. The new service object uses the “auth_token” returned by the OpenStack API and not the provided username and password originally used for authentication.

Example

Requirements

• Ruby (1.8.7, 1.9.2 or 1.9.3)

• Fog (Installed via: gem install fog)

Connection# Connect to Openstack Nova

>> compute = Fog::Compute.new({:provider => 'openstack', :openstack_tenant => 'demo', :openstack_api_key => 'yourpassword', :openstack_username => 'admin', :openstack_auth_url => 'http://localhost:35357/v2.0/tokens'})

#<Fog::Compute::OpenStack::Real:25457260 @openstack_region=nil @openstack_auth_token=nil @openstack_username="admin" @openstack_service_type=["nova", "compute"] @auth_token_expiration="2013-02-21T10:30:10Z" ... snip ... @tenant_id="ee1b6e1715644f2ca020c4190769d496" @openstack_identity_public_endpoint="http://localhost:5000/v2.0" @openstack_management_url="http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496" @port=8774 @scheme="http" @openstack_service_name=nil>

# Query the user>> compute.current_user{"username"=>"admin", "roles_links"=>[], "id"=>"5b772e8e179249aeb7152742a4b4f6e9", "roles"=>[{"name"=>"admin"}, {"name"=>"project_manager"}, {"name"=>"Member"}], "name"=>"admin"}

# Get information about the tenant>> compute.current_tenant{"enabled"=>true, "description"=>"Demo Account", "name"=>"demo", "id"=>"ee1b6e1715644f2ca020c4190769d496"}

# Check the current state of the running instances>> compute.list_servers#<Excon::Response:0x000000030b1a28 @headers={"X-Compute-Request-Id"=>"req-eb228f6f-bec7-4fe3-8d41-69ad69dbee43", "Content-Type"=>"application/json", "Content-Length"=>"15", "Date"=>"Wed, 20 Feb 2013 10:30:13 GMT"}, @status=200, @remote_ip="127.0.0.1", @body={"servers"=>[]}>

Servers# Get a flavor (VM Size) to use for starting a instance

>> flavor = compute.list_flavors.body['flavors'].first{"id"=>"2", "links"=>[{"href"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/flavors/2", "rel"=>"self"}, {"href"=>"http://localhost:8774/ee1b6e1715644f2ca020c4190769d496/flavors/2", "rel"=>"bookmark"}], "name"=>"m1.small"}

# Get an image to use for starting a instance>> image = compute.list_images.body['images'].first{"id"=>"10a11aa4-3d9c-49e5-988c-3fde3cf37842", "links"=>[{"href"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/images/10a11aa4-3d9c-49e5-988c-3fde3cf37842", "rel"=>"self"}, {"href"=>"http://localhost:8774/ee1b6e1715644f2ca020c4190769d496/images/10a11aa4-3d9c-49e5-988c-3fde3cf37842", "rel"=>"bookmark"}, {"href"=>"http://10.50.2.1:9292/ee1b6e1715644f2ca020c4190769d496/images/10a11aa4-3d9c-49e5-988c-3fde3cf37842", "type"=>"application/vnd.openstack.image", "rel"=>"alternate"}], "name"=>"'64Bit_Ubuntu_12.04'"}

# Boot a new server>> compute.create_server('MyFirstServer', image['id'], flavor['id'])#<Excon::Response:0x000000031d3af0 @headers={"X-Compute-Request-Id"=>"req-a0379de9-a129-496b-b627-11853e0eefca", "Location"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/servers/1334c522-ab51-403f-b694-0efd92ef0b10", "Content-Type"=>"application/json", "Content-Length"=>"462", "Date"=>"Wed, 20 Feb 2013 10:36:37 GMT"}, @status=202, @remote_ip="127.0.0.1", @body={"server"=>{"security_groups"=>[{"name"=>"default"}], "OS-DCF:diskConfig"=>"MANUAL", "id"=>"1334c522-ab51-403f-b694-0efd92ef0b10", "links"=>[{"href"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/servers/1334c522-ab51-403f-b694-0efd92ef0b10", "rel"=>"self"}, {"href"=>"http://localhost:8774/ee1b6e1715644f2ca020c4190769d496/servers/1334c522-ab51-403f-b694-0efd92ef0b10", "rel"=>"bookmark"}], "adminPass"=>"maUXeZE76o64"}}>

# Check the current state of running instances>> servers = compute.list_servers.body['servers']#<Excon::Response:0x000000031e16c8 @headers={"X-Compute-Request-Id"=>"req-cae13781-1377-4fc5-a8d2-77394468344d", "Content-Type"=>"application/json", "Content-Length"=>"388", "Date"=>"Wed, 20 Feb 2013 10:37:01 GMT"}, @status=200, @remote_ip="127.0.0.1", @body={"servers"=>[{"id"=>"1334c522-ab51-403f-b694-0efd92ef0b10", "links"=>[{"href"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/servers/1334c522-ab51-403f-b694-0efd92ef0b10", "rel"=>"self"}, {"href"=>"http://localhost:8774/ee1b6e1715644f2ca020c4190769d496/servers/1334c522-ab51-403f-b694-0efd92ef0b10", "rel"=>"bookmark"}], "name"=>"MyFirstServer"}]}>

Model Queries# Example of a Collection query for the same data

>> compute.servers<Fog::Compute::OpenStack::Servers filters={} [ <Fog::Compute::OpenStack::Server id="1334c522-ab51-403f-b694-0efd92ef0b10", instance_name=nil, addresses={"novanetwork"=>[{"version"=>4, "addr"=>"10.2.0.6"}]}, flavor={"id"=>"2", "links"=>[{"href"=>"http://localhost:8774/ee1b6e1715644f2ca020c4190769d496/flavors/2", "rel"=>"bookmark"}]}, host_id="c1e95b9f69157e41baebad3092bd069baaf840215ecbee7edddd9bb2", image={"id"=>"10a11aa4-3d9c-49e5-988c-3fde3cf37842", "links"=>[{"href"=>"http://localhost:8774/../images/...", "rel"=>"bookmark"}]}, metadata= <Fog::Compute::OpenStack::Metadata [] >, links=[{"href"=>"http://localhost:8774/v2/.../servers/...", "rel"=>"self"}, {"href"=>"http://localhost:8774/.../servers/...", "rel"=>"bookmark"}], name="MyFirstServer", personality=nil, progress=0, accessIPv4="", accessIPv6="", availability_zone=nil, user_data_encoded=nil, state="ACTIVE", created=2013-02-20 10:36:37 UTC, updated=2013-02-20 10:37:24 UTC, tenant_id="ee1b6e1715644f2ca020c4190769d496", user_id="5b772e8e179249aeb7152742a4b4f6e9", key_name=nil, fault=nil, os_dcf_disk_config="MANUAL", os_ext_srv_attr_host="cn26.la-1-2.morphlabs.net", os_ext_srv_attr_hypervisor_hostname="cn26.la-1-2.morphlabs.net", os_ext_srv_attr_instance_name="instance-00000038", os_ext_sts_power_state=1, os_ext_sts_task_state=nil, os_ext_sts_vm_state="active" > ] >

Volumes# Create a 1GB persistent volume

>> compute.create_volume("MyFirstVolume", "Test Volume", 1000)# <Excon::Response:0x00000003257940 @headers={"X-Compute-Request-Id"=>"req-e98c1f91-d2b9-49cc-bf14-353fd8fa46be", "Location"=>"http://localhost:8774/v2/ee1b6e1715644f2ca020c4190769d496/os-volumes?ignore_awful_caching1361357305/664000aa-e275-4e86-a477-7be86927979b", "Content-Type"=>"application/json", "Content-Length"=>"311", "Date"=>"Wed, 20 Feb 2013 10:48:27 GMT"}, @status=200, @remote_ip="127.0.0.1", @body={"volume"=>{"status"=>"creating", "displayDescription"=>"Test Volume", "availabilityZone"=>"nova", "displayName"=>"MyFirstVolume", "attachments"=>[{}], "volumeType"=>nil, "snapshotId"=>nil, "metadata"=>{}, "id"=>"664000aa-e275-4e86-a477-7be86927979b", "createdAt"=>"2013-02-20T10:48:26.709904", "size"=>1000}}>

# Query information about the current volumes for this tenant>> compute.list_volumes.body['volumes'].first{"status"=>"available", "displayDescription"=>"Test Volume", "availabilityZone"=>"nova", "displayName"=>"MyFirstVolume", "attachments"=>[{}], "volumeType"=>nil, "snapshotId"=>nil, "metadata"=>{}, "id"=>"664000aa-e275-4e86-a477-7be86927979b", "createdAt"=>"2013-02-20T10:48:26.000000", "size"=>1000}

# Attach the Volume to the running server>> compute.attach_volume(compute.list_volumes.body['volumes'].first['id'], compute.servers.first.id, "vdc")#<Excon::Response:0x000000032b0680 @headers={"X-Compute-Request-Id"=>"req-b6a2bad1-b69d-4abc-a3e6-88d1fc044827", "Content-Type"=>"application/json", "Content-Length"=>"194", "Date"=>"Wed, 20 Feb 2013 10:54:12 GMT"}, @status=200, @remote_ip="127.0.0.1", @body={"volumeAttachment"=>{"device"=>"/dev/vdc", "serverId"=>"1334c522-ab51-403f-b694-0efd92ef0b10", "id"=>"664000aa-e275-4e86-a477-7be86927979b", "volumeId"=>"664000aa-e275-4e86-a477-7be86927979b"}}>

# Check volumes by querying the Volumes collection>> compute.volumes.first <Fog::Compute::OpenStack::Volume id="664000aa-e275-4e86-a477-7be86927979b", name="MyFirstVolume", description="Test Volume", status="in-use", size=1000, type=nil, snapshot_id=nil, availability_zone="nova", created_at="2013-02-20T10:48:26.000000", attachments=[{"device"=>"/dev/vdc", "serverId"=>"1334c522-ab51-403f-b694-0efd92ef0b10", "id"=>"664000aa-e275-4e86-a477-7be86927979b", "volumeId"=>"664000aa-e275-4e86-a477-7be86927979b"}] >

Cleanup

# Remove the Volume from the instance>> compute.detach_volume(compute.servers.first.id, compute.list_volumes.body['volumes'].first['id'])#<Excon::Response:0x0000000338e278 @headers={"Content-Type"=>"text/html; charset=UTF-8", "Content-Length"=>"0", "Date"=>"Wed, 20 Feb 2013 10:58:10 GMT"}, @status=202, @remote_ip="127.0.0.1", @body="">

# Delete the Volume from the system>> compute.delete_volume(compute.list_volumes.body['volumes'].first['id'])#<Excon::Response:0x0000000339f6b8 @headers={"Content-Type"=>"text/html; charset=UTF-8", "Content-Length"=>"0", "Date"=>"Wed, 20 Feb 2013 10:59:22 GMT"}, @status=202, @remote_ip="127.0.0.1", @body="">

# Check volumes again>> compute.list_volumes.body['volumes']# []

# Cleanup the system by deleting the instancecompute.delete_server(servers.first['id'])#<Excon::Response:0x000000033b9dd8 @headers={"Content-Length"=>"0", "X-Compute-Request-Id"=>"req-ef6c799f-7832-4edf-9929-387d5957320a", "Content-Type"=>"application/json", "Date"=>"Wed, 20 Feb 2013 11:00:17 GMT"}, @status=204, @remote_ip="127.0.0.1", @body="">

Thank you!