CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.* * All unlicensed or borrowed works retain their original licenses. Pimp My Cloud Nova Configuration Hints and Tricks Joe Gordon 1 Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Pimp My CloudNova Configuration Hints and Tricks
Joe Gordon
1
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
About Me
• Engineer at Cloudscaling• Contributor• Deployer• Folsom Contributions
o Top 10 developer (by commits)
o Mostly in Nova
http://bitergia.com/public/reports/openstack/2012_09_folsom/
2
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
By the Numbers: Nova Folsom
• 190+ Contributors •
• Code churn:
Release Python Lines Other Lines Python Files Other Files
Folsom 186,738 242,721 666 788
Essex 150,894 221,109 593 302
Diablo 110,581 110,393 427 389
Code churn generated with git log --numstat --pretty="%H" $A..$B| grep \.py$ | awk 'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%d\n", plus, minus)}'
Release Lines Insertions Lines Deletions Insertions/LoC %
Folsom 110,308 71,911 59.0%
Essex 182,298 138,346 120.8%
3
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Who Wrote Nova Folsom?
4
1 138 Russell Bryant <[email protected]> 2 112 Johannes Erdfelt <[email protected]> 3 97 Dan Prince <[email protected]> 4 88 Vishvananda Ishaya <[email protected]> 5 81 Joe Gordon <[email protected]> 6 63 Michael Still <[email protected]> 7 59 Mark McLoughlin <[email protected]> 8 58 Rick Harris <[email protected]> 9 50 Yun Mao <[email protected]> 10 45 Daniel P. Berrange <[email protected]> 11 36 Chris Behrens <[email protected]> 12 31 Eoghan Glynn <[email protected]> 13 29 Brian Waldon <[email protected]> 14 26 Pádraig Brady <[email protected]> 15 25 Chuck Short <[email protected]> 16 23 Sean Dague <[email protected]> 17 21 Alex Meade <[email protected]> 18 18 Kevin L. Mitchell <[email protected]> 19 17 Brian Elliott <[email protected]> 20 17 Zhongyue Luo <[email protected]> 21 16 John Griffith <[email protected]> 22 13 Dan Smith <[email protected]> 23 13 Andrew Bogott <[email protected]> 24 12 Renuka Apte <[email protected]> 25 12 Thierry Carrez <[email protected]> 26 12 Monty Taylor <[email protected]> 27 10 MotoKen <[email protected]>
git shortlog -sne --since="Tue Mar 20 08:17:40 2012 +0100" --no-merges | cat -n
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
OpenStack Architecture
5
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Nova Architecture
6
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Vanilla Nova
• Devstack• All in One• RabbitMQ• KVM• MySQL• Default configuration options
7
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Customize Nova
• Choose ao Hypervisoro Databaseo Message Queueo Network Modelo Scheduler
• Configurationo Any of the 500+ options
• Custom Serviceso Any service can be
swapped out for one you write
8
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Customize Nova
• Choose ao Hypervisoro Databaseo Message Queueo Network Modelo Scheduler
• Configurationo Any of the 500+ options
• Custom Serviceso Any service can be
swapped out for one you write
9
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
What Makes Openstack Open
• 3 RPC backends• 3+ DB backends• 6+ Virtualization backends• 500+ configuration options
10
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
New Options for Folsom
• 1 RPC backendso ZeroMQ
• 2 virt backendso Hyper-Vo PowerVM
• 100+ configuration options
11
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
New Features for Folsom by Blueprint
• General host aggregates (operator)• Disable API extensions• Rootwrap pluggable filters (operator)• Multi-scheduler support (operator)• Flavor extra-specs• LVM ephemeral disk images (operator)• Project specific flavors• Multi-process API services (operator)
12
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Backend Choices
13
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
RPC Comparisons
RPC Backend Protocol Company Affiliations Brokerless Language
kombu(rabbitMQ) AMQP 0-9-1 Erlang
qpid AMQP 0-10 Java, C++
ZeroMQ Python, C++
14
Wednesday, October 17, 12
Database Simultaneous writes High Availability Native python client
?
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Database Comparisons
15
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Virtualization Comparisons
16
Backend maintained by Virtualization type Fast Open Source
Baremetal N/A N/A
UML paravirt only
LXC container
Xen full/paravirt
KVM full
Qemu qemu.org full
Hyper-V full
PowerVM full
VMware ESX full
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Configuration Options
17
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Configuration Options by Type
18
57%
1%4%
23%
1%
13%
Boolean 68Floats 6Ints 119Lists 18MultiString 6String 290
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
New Config Options in Folsomallowed_rpc_exception_modulesapi_urlauth_blobbackdoor_portbase_dir_namecinder_*claim_timeout_secondsconfig_drive_formatconfig_drive_skip_versionsconfig_drive_tempdirdefault_scheduler_driverdefer_iptables_applyec2_strict_validationec2_workersfatal_deprecationsforce_config_driveglance_api_insecurehyperv_attaching_volume_retry_counthyperv_wait_between_attach_retryimage_info_filename_patterninstance_build_timeoutinstance_uuid_formatlibvirt_cpu_modelibvirt_cpu_modellibvirt_images_typelibvirt_images_volume_grouplibvirt_inject_keylibvirt_inject_partitionlibvirt_snapshots_directorylibvirt_sparse_logical_volumeslimit_cpu_featuresmatchmaker_ringfile
max_agenetapp_storage_service_prefixnfs_mount_point_basenon_inheritable_image_propertiesnotify_on_any_changenotify_on_state_changeosapi_max_request_body_sizeosapi_volume_workersperiodic_fuzzy_delayportpowervm_*quantum_*quota_driverquota_key_pairsquota_security_group_rulesquota_security_groupsrbd_*reservation_expirerootwrap_configrpc_cast_timeoutrpc_zmq_*serverserver_ca_filesnapshot_same_hostsql_connection_debugsql_connection_tracestorwize_*syslog_log_facilityuntil_refreshvswitch_namexenapi_check_hostxenapi_num_vbd_unplug_retries
19
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Virtualization options1.pick a compute RPC topic scheduler_topic=compute
2.pick a compute managercompute_manager=nova.compute.manager.ComputeManager
3.compute manager optionsinstances_path=$state_path/instancesbase_dir_name=_basecompute_driver=libvirt.LibvirtDriverconsole_host=socket.getfqdn()live_migration_retry_count=30reboot_timeout=0instance_build_timeout=0rescue_timeout=0resize_confirm_window=0*_interval=running_deleted_instance_timeout=0running_deleted_instance_action=loginstance_usage_audit=false
4.compute driver optionsrescue_*_id=libvirt_type=kvmlibvirt_inject_password=falselibvirt_inject_key=true*_migration_=libvirt_cpu_mode=libvirt_cpu_model=...
20
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
compute.manager options cfg.IntOpt('live_migration_retry_count',
default=30, help="Number of 1 second retries needed in live_migration"), cfg.IntOpt("reboot_timeout", default=0, help="Automatically hard reboot an instance if it has been " "stuck in a rebooting state longer than N seconds. " "Set to 0 to disable."), cfg.IntOpt("instance_build_timeout", default=0, help="Amount of time in seconds an instance can be in BUILD " "before going into ERROR status." "Set to 0 to disable."), cfg.IntOpt("rescue_timeout", default=0, help="Automatically unrescue an instance after N seconds. " "Set to 0 to disable."), cfg.IntOpt("resize_confirm_window", default=0, help="Automatically confirm resizes after N seconds. " "Set to 0 to disable."), ... cfg.StrOpt("running_deleted_instance_action", default="log", help="Action to take if a running deleted instance is detected." "Valid options are 'noop', 'log' and 'reap'. " "Set to 'noop' to disable."),
21
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
reap running deleted instances cfg.IntOpt("running_deleted_instance_timeout",
default=0, help="Number of seconds after being deleted when a running " "instance should be considered eligible for cleanup."), cfg.IntOpt("running_deleted_instance_interval", default=30, help="Number of periodic scheduler ticks to wait between runs of " "the cleanup task."), cfg.StrOpt("running_deleted_instance_action", default="noop", help="Action to take if a running deleted instance is detected." "Valid options are 'noop', 'log' and 'reap'. " "Set to 'noop' to disable."),
nova.conf settings
running_deleted_instance_timeout=60running_deleted_instance_action=reap
22
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Scheduler options1.pick a scheduler RPC topic scheduler_topic=scheduler
2.pick a scheduler managerscheduler_manager=nova.scheduler.manager.SchedulerManager
3.pick a scheduler driverscheduler_driver=nova.scheduler.multi.MultiScheduler
4.set up scheduler.driver optionsscheduler_host_manager=nova.scheduler.host_manager.HostManagerscheduler_max_attempts=3
5.multi scheduler optionscompute_scheduler_driver=nova.scheduler.filter_scheduler.FilterSchedulervolume_scheduler_driver=nova.scheduler.chance.ChanceSchedulerdefault_scheduler_driver=nova.scheduler.chance.ChanceScheduler
6.host_manager optionsscheduler_available_filters=nova.scheduler.filters.standard_filtersscheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter
7.filter optionsram_allocation_ratio=1.5
8.resource_tracker optionsreserved_host_disk_mb=0reserved_host_memory_mb=512claim_timeout_seconds=600compute_stats_class=nova.compute.stats.Stats
23
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Add custom filtercfg.MultiStrOpt('scheduler_available_filters',
default=['nova.scheduler.filters.standard_filters'], help='Filter classes available to the scheduler which may ' 'be specified more than once. An entry of ' '"nova.scheduler.filters.standard_filters" ' 'maps to all filters included with nova.'), cfg.ListOpt('scheduler_default_filters', default=[ 'RetryFilter', 'AvailabilityZoneFilter', 'RamFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter' ], help='Which filter class names to use for filtering hosts ' 'when not specified in the request.'),
nova.conf settings
scheduler_available_filters=myfilter.CustomFilterscheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter, ComputeCapabilitiesFilter,ImagePropertiesFilter,CustomFilter
24
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
*_driver *_manager
• network• quota• compute• console• db• linuxnet_interface• scheduler
o compute_schedulero volume_schedulero default_scheduler
• baremetal• firewall• libvirt_vif• volume_driver
• compute• console• cert• instance_dns• floating_ip_dns• network• volume• scheduler• metadata• image_cache• consoleauth• scheduler_host
25
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Nova Architecture
26
Driver
manager
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
ResourcesBlueprints: https://blueprints.launchpad.net/nova/folsomSource: https://github.com/openstack/nova/tree/stable/folsomDocumentation: http://docs.openstack.org Release notes: http://wiki.openstack.org/ReleaseNotes/Folsom
27
Wednesday, October 17, 12
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.** All unlicensed or borrowed works retain their original licenses.
Questions?
28
Wednesday, October 17, 12