OpenStack Acceleration As A Service Introducing OpenStack Cyborg
OpenStack Acceleration As A
Service
Introducing OpenStack Cyborg
Bio
● Open Source Community Manager
○ OpenStack Cyborg PTL, Interop WG Contributor, Public Cloud WG co-chair
○ OpenSDS Core Contributor
○ OPNFV committer for several projects
○ Hyperleger Contributor
● Standard Engineer
○ Rapporteur for ETSI NFV IFA 004 & 019Wechat & Line: +8618576658966
Email: [email protected]
Outline
● Background
● OpenStack HPC from Scientific WG’s perspective
● Introduction of Cyborg Project
● Future Works
“as more and more workloads migrate to the
cloud, the mass concentration of similar
workloads make hardware acceleration a
requirement rather than an interesting option”
——James Hamilton
http://perspectives.mvdirona.com/2017/04/ten
sor-processing-unit/
Background
Background● OpenStack Acceleration Discussion Started from Telco Requirements
○ High level requirements first drafted in the standard organization ETSI NFV ISG
○ High level requirements transformed into detailed requirements in OPNFV DPACC project.
○ New project called Nomad established to address the requirements.
○ BoF discussions back in OpenStack Austin Summit.
● Transition to Cyborg Project
○ From a long period of conversation and discussion within the OpenStack community, we found that the initial goal of Nomad project to address acceleration management in Telco is too limited. From design summit session in Barcelona Summit, we have developers from Scientific WG help us understanding the need for acceleration management in HPC cloud, and we also had a lot of discussion on the Public Cloudsupport of accelerated instances.
○ We decide to formally establish a project that will work on the management framework for dedicated devices in OpenStack, and there comes the Cyborg Project.
Background
● Cyborg Project Quick Facts
○ Started Dec 2016, not an official project yet
○ Development start from zero in OpenStack, follow “four opens” principle
○ Weekly meeting UTC 1500 on Wed at #openstack-cyborg, minutes at http://eavesdrop.openstack.org/meetings/openstack_cyborg/
○ Use openstack-dev mailing list with [acceleration]
○ Wiki at https://wiki.openstack.org/wiki/Cyborg
○ Stats : http://stackalytics.com/?project_type=openstack-others&module=cyborg&metric=person-day
○ Code: https://review.openstack.org/#/q/project:openstack/cyborg
Outline
● Background
● OpenStack HPC from Scientific WG’s perspective
● Introduction of Cyborg Project
● Future Works
Current issues using GPUs in Openstack
GPUs can be used in openstack. GPU specific flavors: pci_passthrough alias in the properties field
KVM tuning is required to achieve acceptable performance
Two options:Heterogeneous hosts: GPU and CPU-only hosts mixed
Bad - scheduler does not prioritize GPU workloads.
Good - CPU resources are available for all workloads
GPU only Host-Aggregate: GPU hosts are segregated
Good - GPU hosts are only used for GPU workloads
Bad - CPU-only workloads unable to use underutilized GPU hosts
Outline
● Background
● OpenStack HPC from Scientific WG’s perspective
● Introduction of Cyborg Project
● Future Works
What is Cyborgcyborg-api
cyborg-agent
cyborg-db
local cache
cyborg-generic-drivervendor-a-driver vendor-b-driver
vendor-a-fpga vendor-c-ipsec vendor-b-gpgpu
Control Node
Compute Node
cyborg-conductor
nova-api
What will Cyborg do with Nova?
cyborg-db
cyborg-api
cyborg-agent
placement-api
Acc driver
nova-compute
Libvirt driver
nova-conductor
nova-db
cyborg-conductor nova-scheduler
Outline
● Background
● Introduction of Cyborg Project
● OpenStack HPC from Scientific WG’s perspective
● Future Works
Future Work
Cyborg Development Goal:
• Finish the code development for the initial specs in Pike
• Have more contributors (especially from vendors to help us understand real case)
• Fulfill requirements from SWG and other related WGs
• Looking for PoC opportunities
• FPGA and GPGPU basic support in Queen
Future Work
Cross Community Efforts:
• Initial Container Device Interface (CDI) proposal in Kubernetes:
https://github.com/kubernetes/community/pull/545/
• OpenCL
• OPNFV DPACC project