Configure your cloud to be able to charge your users using official OpenStack components! 27 octobre 2016 Ref 20160424-ObjectifLibre- HandsOntoConfigureYourCloudtoBeAbletoChargeYourUsersUsingOfficialOpenStackComponents Objectif Libre http://www.objectif-libre.com [email protected]
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Configure your cloud to be able to chargeyour users using official OpenStack
Configure your cloud to be able to charge your users using official OpenStack components!
∙ Let’s meet your hosts
∙ Your infrastructure !
∙ All you need is good tools !
∙ Our setup
∙ Let’s cook gnocchis !
∙ Collecting the metrics
∙ Cloudkitty’s turn
∙ Here comes the pricing policy
∙ Reporting
∙ It is time for questions
Table des matières 1
Let’s meet your hosts !
∙ Stéphane Albert (Objectif Libre)Co-father of CloudKitty / Former CloudKitty PTL
∙ Christophe Sauthier (Objectif Libre)CEO of Objectif Libre / PTL and co-Father of CloudKitty
∙ Julien Danjou (Red Hat)PTL Telemetry / Father of Gnocchi
∙ Maxime Cottret (Objectif Libre)Cloud Consultant / CloudKitty Contributor
∙ Luka Peschke (Objectif Libre)DevOps trainee / Cloudkitty Contributor
2
Today’s plate !
Start by getting your image
Go to http ://olib.re/handson-bcn and save it !
Grab an IP
Go to https ://quotidien.framapad.org/p/osa-b and put your name next to an IP
3
Your infrastructure !
What’s to expect in that image ?
Some essential OpenStack components :
∙ A mariadb database.
∙ An AMQP (rabbitmq)
∙ memcached
∙ Keystone bootstrapped with an user (admin, password : adminpass) on a single project (admin).
∙ Glance
∙ Nova
∙ Horizon
∙ Neutron
Login : root / password : root
* PLEASE CHANGE IT * (using passwd once loggued by exemple)
4
All you need is good tools !
Ceilometer
OpenStack measurement project
Ceilometer (Telemetry) collects the usage of all the resources in an OpenStack Cloud.
Those informatios are stored in databases, and a REST API is provided to access them.
Some advanced capabilities (like alarming with aodh)
ArchitectureCeilometer is composed of various parts, the main ones are :
∙ ceilometer-api (controler) : API server∙ ceilometer-collector (controler) : reads AMQP messages (from other components like nova)∙ ceilometer-agent-central (controller) : polls some metrics directly∙ ceilometer-agent-compute (compute node) : fetch informations related to instances (like CPU
usage)
MetricsMetrics are the elements storing cloud usage, like :
∙ cpu_util : CPU usage percentage∙ network.incoming.bytes : number of octets received on a network port∙ disk.read.bytes : number of bytes read from a disk
Metrics can be of different types :∙ gauge : value at the precise moment of collection∙ cumulative : sum of all the previous values∙ delta : difference since the last value
Samplessamples are the basis of ceilometer calculations : raw data stored after each polling or message received.
Each sample is composed of a record date, a name, a type, a value and an unit.
Scalability issueA scalability issue raised due to the following reasons :
∙ Many requests done by API requires full scan of all stored samples∙ one can filter on any fields∙ metadata are not indexed (since free-form)
Some improvements have been tried, but many felt it wasn’t the good approach...
5
Configure your cloud to be able to charge your users using official OpenStack components!
Gnocchi
TDBaaS (Time Series Database as a Service) project started under the Ceilometer program um-brella.
The idea is to do aggregation of samples over a period of time to only store a reduced amount of data.
Gnocchi can be used as a storage backend for Ceilometer, and since it provides its own API some third partysoftware can uses the mecanism avoiding the scalability issue of Ceilometer.
Archive PolicyThe number of samples can be different over the time
∙ 1 every second over the last day∙ 1 every minute for the last week∙ 1 every hour for the last 6 months
The indexerGnocchi supports many indexer (Postgres, Mysql) that are simply in charge of the relationship between metrics andresources.
Every known OpenStack resources have a model, dropping the free-form metadata that were used in Ceilometer(and thus improving speed/scalability)
CloudKitty
An Open Source rating/chargeback component for OpenStack∙ Official component since Sept 2015∙ Developped in python with all OpenStack best practices∙ Integrated with Horizon∙ Interfaced with both Ceilometer and Gnocchi∙ Uber Modular
Modularity4 Levels of modularity
∙ Collection of usage∙ Pricing Policy∙ Output format∙ Output Storage
A Closer look at the Hashmap moduleAims at matching arbitrary values from a service type to resource metadata
∙ Group : Define set of rules that are processed together∙ Service : Match a CK Service (compute, network,..)∙ Field : Match a resource metadata field (flavor, image_id,...)∙ Mapping
∘ Can be attached to a service or a field∘ Map the value of a metadata to a cost and a type of cost
∙ Threshold∘ Can be attached to a service or a field∘ Define thresold levels on a metadata to a cost
4.2. Gnocchi 6
Today’s setup
What do we use ?
We are going to use Ceilometer/Gnocchi to collect metrics, CloudKitty for the rating, and Gnocchi to store them.
Why do we need Ceilometer and Gnocchi ?
Ceilometer is able to collect AND to store metrics. However, Gnocchi’s storage is much more efficient, that’s whywe use it.
7
Let’s cook gnocchis !
Who are you ?
First of all, export some environment variables needed for keystone authentication :
Configure your cloud to be able to charge your users using official OpenStack components!
mysql -uroot -pmysqlpass << EOFCREATE DATABASE gnocchi;GRANT ALL PRIVILEGES ON gnocchi.* TO 'gnocchi'@'localhost' \IDENTIFIED BY 'gnocchidbpassword';GRANT ALL PRIVILEGES ON gnocchi.* TO 'gnocchi'@'%' \IDENTIFIED BY 'gnocchidbpassword';EOF
Field Valuedomain_id id name None 66af44925dfd403e8027ba89fdcd48e1 rating
Creating cloudkitty user
openstack user create --project service --password password cloudkitty
Field Valuedefault_project_id f4a80b4777554cbcaa3074c95be7fe3edomain_id defaultenabled Trueid 3a524d3f98cb49cea49d46e73433299cname cloudkitty
openstack role add --user cloudkitty --project service admin
Creating the CloudKitty database
mysql -uroot -pmysqlpass << EOFCREATE DATABASE cloudkitty;GRANT ALL PRIVILEGES ON cloudkitty.* TO 'cloudkitty'@'localhost' \IDENTIFIED BY 'ckdbpassword';
cloudkitty-dbsync upgrade| 2016-10-13 02:24:13.908 2816 DEBUG oslo_db.api [-] Loading backend 'sqlalchemy' from 'cloudkitty.db.sqlalchemy.api' _load_backend /usr/lib/python2.7/site-packages/oslo_db/api.py:234| INFO [alembic.runtime.migration] Context impl MySQLImpl.| INFO [alembic.runtime.migration] Will assume non-transactional DDL.| INFO [alembic.runtime.migration] Running upgrade -> 464e951dc3b8, Initial migration| INFO [alembic.runtime.migration] Running upgrade 464e951dc3b8 -> 2ac2217dcbd9, Added support for meta collector| INFO [alembic.runtime.migration] Running upgrade 2ac2217dcbd9 -> 385e33fef139, Added priority to modules_state.| INFO [alembic.runtime.migration] Context impl MySQLImpl.| INFO [alembic.runtime.migration] Will assume non-transactional DDL.| INFO [alembic.runtime.migration] Running upgrade -> 3dd7e13527f3, Initial migration| INFO [alembic.runtime.migration] Running upgrade 3dd7e13527f3 -> 4fa888fd7eda, Added threshold support.| INFO [alembic.runtime.migration] Running upgrade 4fa888fd7eda -> 54cc17accf2c, Fixed constraint name.| INFO [alembic.runtime.migration] Context impl MySQLImpl.| INFO [alembic.runtime.migration] Will assume non-transactional DDL.| INFO [alembic.runtime.migration] Running upgrade -> 4f9efa4601c0, Initial migration.
Setting up the storage
cloudkitty-storage-init| INFO [alembic.runtime.migration] Context impl MySQLImpl.| INFO [alembic.runtime.migration] Will assume non-transactional DDL.| INFO [alembic.runtime.migration] Running upgrade -> 4c2f20df7491, Initial migration.
Restarting the cloudkitty-api
systemctl restart cloudkitty-api
Setting up rating rules
List available rating modules :
cloudkitty module-list
8.4. Configuring CloudKitty 14
Configure your cloud to be able to charge your users using official OpenStack components!
Configure your cloud to be able to charge your users using official OpenStack components!
9.4. Let’s use Horizon for the policy ! 17
Configure your cloud to be able to charge your users using official OpenStack components!
9.4. Let’s use Horizon for the policy ! 18
Configure your cloud to be able to charge your users using official OpenStack components!
∙ Group : select the group you just created (instance_uptime)
That’s it ! You have defined a rule to charge your users depending on the used flavor. Let’s create a second one witha different flavor id : barflavor.
Let’s fill some data
You can wait here for hours to use your “Cloud” or use the script :)
Configure your cloud to be able to charge your users using official OpenStack components!
Predictive Pricing
openstack flavor list
Create a new mapping in the flavor_id field use the id of the m1.nano flavor as value.
Create an image_id field, and create a new mapping inside it with the ID of the glance image myimage thesame way you added the m1.nano flavor. Decide that it is a rate and it should be 10% more expensive than theusual price.
Look at the predictive pricing !Launch a few instances with the m1.nano flavor, and wait for a few minutes.