8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 1/18
1
KVM PV DEVICES
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 2/18
2
Agenda
Introduction & brief history
VirtIO
Enhanced VirtIO with KVM support
Further implementation
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 3/18
3
General & history
Fully virtualized devices performs bad
55 Mbps for RTL
Lots of io-exits per packet
Decided to implement a modern e1000
Advantage:
Only Qemu coding no guest tools involved
Irq coalescing
Only 2-3 io-exits per packet
Can be the base of user-space PV
But then came Ingo…
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 4/18
4
PV driver architecture
Linux kernel & KVM
KVMPV backend
driver
VM
VM
Qemu
HVM guest
PVfrontend
driver
PVbackend
driver
RING
VM
Qemu
HVM guest
PV
frontenddriver
PVbackend
driver
RIN
G
VM
Qemu
HVM guest
PVfrontend
driver
PVbackend
driver
RI
NG
VM
Qemu
HVM guest
PVfrontend
driver
PV
backenddriver
R
ING
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 5/18
5
General & history
V0 - leveraging Ingo Molnar’s PV code
Make loadable module
Add HVM support
Add NAPI
Add memory barriers and improved ring
Keep running after performance & stability
Merge to the kernel?
Alternatives
Xen
Polished drivers Xen specific
VirtIO was just published.
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 6/18
6
PV driver requirements
Close to native performance
Merge with the kernel
Leverage existing code
Have usermode implementation Block device for qcow, vmdk formats
Ability to function without KVM (-no-kvm)
Ability to run HVM guests
Robust code
Easily add drivers
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 7/18
7
VirtIO
An API for virtual I/O
Implements network & block driver logic
Written by Rusty Russell
Motivation
Many hypervisors of all types
Hard to tune and maintain each one Code reuse – The KVM way ;)
Implementations
Lguest
KVM
Possible (Xen, UML, Qemu, VMware?..)
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 8/18
8
Linux kernel & KVM
KVM PV backend
driver
VM
VM
Qemu
VIRTIO
NET
Guest
kernel
VIRTIO BlK
Hypervisor Specifics
PV code
VirtIO
scatterlist sg[] for skb/blk_req data
Add_bufGet_buf Sync
Kick
Restart Detach
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 9/18
9
VirtIO – Hypervisor specifics
The front end logic is implemented by VirtIO
The backend needs
Probing & Bus services
Enumeration
Irq
Parameters (mac,..)
Shared memory with remote side
Hypercalls
Host driver/userspace backend
Linux kernel & KVM
KVM PV backenddriver
VM
VM
Qemu
VIRTIONET
Guestkernel
VIRTIO BlK
Hypervisor Specifics
PV code
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 10/18
10
Enhanced VirtIO
Motivation
Increase re-use
Allow operation with various bus types Make new devices code smallest
Components Shared memory code
With per hypervisor I/O hypercalls
Bus (pci, virtual bus)
Host backend
Linux kernel & KVM
KVM PV backenddriver
VM
VM
Qemu
VIRTIO
NET
Guest
kernel
VIRTIO BlK
Hypervisor SpecificsPV code
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 11/18
11
Enhanced VirtIO
Linux kernel & KVM
KVMPV
backenddriver
VM
VM
Qemu
Guest
kernel
VIRTIO BlK
VIRTIO
Virtual Bus
Sharedmem
backendHypercalls
Enhanced VIRTIO
VIRTIO NetNet
Backend
BlkBackend
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 12/18
12
Enhanced VirtIO
Status:
Interface needs polishing
KVM support PCI bus support
Result: Makes backend driver tiny
620 Mbps throughput for
networkHVM Linux guest
Before optimization
Userspace backend driver
Linux kernel & KVM
KVM PVbackend
driver
VM
VM
Qemu
Guestkernel
VIRTIO BlK
VIRTIO
Virtual Bus
Sharedmem
backendHypercalls
Enhanced VIRTIO
VIRTIO NetNetBackendBlkBackend
162 LOC 195 LOC
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 13/18
13
Enhanced VirtIO – shared memory backend
VirtIO backend
Implements VirtIO interface
Callbacks to hypervisor and
Code consists of
add_buf, get_buf, restart, detach_buf
Only shared memory logic needed
sync
Ring logic
IO pending hypercall be_virtqueue_interrupt handler
Linux kernel & KVM
KVMPV
backenddriver
VM
VM
Qemu
Guestkernel
VIRTIO BlK
VIRTIO
Virtual Bus
Sharedmem
backendHypercalls
Enhanced VIRTIO
VIRTIO NetNet
Backend
BlkBackend
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 14/18
14
Linux kernel & KVM
KVMPV
backenddriver
VM
VM
Qemu
Guestkernel
VIRTIO BlK
VIRTIO
Virtual Bus
Sharedmem
backendHypercalls
Enhanced VIRTIO
VIRTIO NetNet
Backend
BlkBackend
Enhanced VirtIO – Day in a life of packet
1.Userwrite
2.Kernel
Tcp/ip3.start_xmit
4.add_buf
5.add_buf
6.sync
7.sync
8.Notify hcall
9.The pkts travels to
tap/backend host device
10.Trigger irq
11.irq
12. skb_recv_done
13. get_buf…
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 15/18
15
Enhanced VirtIO – shared memory details
Based on lguest
1-1 shared memory
Data structure
Page of descriptors for rx, tx.
Available pointers page controlled by guest
Used pointers page controlled by host
SG list is currently internal to descriptors
Descriptors are chained by next pointer
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 16/18
16
Enhanced VirtIO – network be driver
Implements kvm_virtnet_probe for pci bus Creates tx,rx be_new_virtqueue
Probes virtnet
Request_irq Irq# taken from bus
Register hypercall - shared memory pfn
Device key for enumeration taken from bus
Linux kernel & KVM
KVMPV
backenddriver
VM
VM
Qemu
Guestkernel
VIRTIO BlK
VIRTIO
Virtual Bus
Shared
membackend
Hypercalls
Enhanced VIRTIO
VIRTIO NetNet
Backend
BlkBackend
195 LOC
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 17/18
17
Further work
Basic
Add readv/writev handlers to Qemu
Complete the user-space block device
Update with VirtIO gso.
Complete migration support
Advanced (also simple) Publish the enhanced interface
Optimize and stabilize
Add host back end drivers
Add virtual bus HVM improvements
Test with PV kernel
Linux kernel & KVM
KVMPV
backenddriver
VM
VM
Qemu
Guestkernel
VIRTIO BlK
VIRTIO
Virtual Bus
Shared
membackend
Hypercalls
Enhanced VIRTIO
VIRTIO NetNet
Backend
BlkBackend
8/6/2019 KvmForum2007$Kvm Pv Drv
http://slidepdf.com/reader/full/kvmforum2007kvm-pv-drv 18/18
18
Thank you ;)
Q&A