The kvm virtualization way

KVM: Virtualisation The Linux Way

Amit [email protected]

GEEP

Copyright © 2007 Qumranet, Inc. All rights reserved.

Virtualisation Strategies

“Native” HypervisorsHave a runtimeNeed a “primary” guest OSExamples: Xen, VMWare ESX Server, IBM mainframes

ContainersDifferent namespaces for different guestsRun on host kernelUserland can be different from hostExamples: OpenVZ, FreeVPS, Linux-Vserver

Paravirtualisation

EmulationExamples: QEMU, PearPC

Copyright © 2007 Qumranet, Inc. All rights reserved.3

KVM: Architectures Supported

S390IBM mainframes: a hypervisor is a mustIncluded in 2.6.26

IA-64Included in 2.6.26

X86Included in 2.6.20KVM-lite: PV Linux guest on non-VTx / non-SVM host (proposed)

PowerPCPVArchitecture support for hypervisorIncluded in 2.6.26


X86 Hardware Extensions

'guest mode' in addition to user and kernel modes

Raise a trap for all privileged instructions

Virtualised registers

ProcessorIntel-VTx (VMX)AMD-V (SVM)

MMEPT (Intel)NPT (AMD)

IOVT-d (Intel)IOMMU (AMD)


What's handled in the kernel?

CPU virtualisation (special instructions)MMU virtualisationLocal APIC, PIC, IOAPIC, PIT(guest) paravirtualised network and block device drivers

virtio-netvirtio-block

(guest) paravirtualised kernel support codeparavirt_opsMMU

(guest) paravirtualised clock driver


KVM Process Model

kernel

task task guest task task guest


KVM Process Model (cont'd)

Guests are scheduled as regular processeskill(1), top(1) work as expectedGuest physical memory is mapped into the task's virtual memory spaceVirtual processors in one VM are threads


KVM Execution Model

Native GuestExecution

KernelExit Handler

UserspaceExit Handler

Switch toGuest Mode

ioctl()

Userspace Kernel Guest

Lightweight ExitHeavyweight Exit


Flow Example: MemoryAccess

Guest accesses an unmapped memory locationHardware traps into kernel modekvm walks the guest page table, determines guest physical addresskvm performs guest physical -> host physical translationkvm installs shadow page table entry containing guest virtual -> host physical translationProcessor restarts execution of faulting instruction


Paravirtualisation

Modifying guest OS for performanceVirtio

Common drivers for all hypervisorsHypervisor-specific backendKVM backend in qemuFaster performanceEfficient block, net driversBalloonlguest, KVM use it already

PV DMAPass through Ethernet devices

paravirt_ops


Network Devices

Fully virtualised device performance not great55 Mbps for RTLLots of IO-exits per packet

Decided to implement a modern e1000Advantages:

All code in userspace (qemu) All existing drivers recognise device

IRQ coalescingOnly 2-3 IO-exits per packetGoes in excess of 800 Mbps


Virtio Net

Shared memory between host and guestTwo queues: recv and sendRing buffer within each queue'available' pointer controlled by guest'used' pointer controlled by host


Virtio-net on KVM

Guest kernel

BLK NET

Virtio PCI

QEMU

Linux

SharedMemory

User-space


Ideas

Shared memory between host and guest via virtio-pciShared directory between host and guest using virtio + fuseVMGL (OpenGL for Virtual Machines) supporthttp://kvm.qumranet.com/kvmwiki/TODO

http://kvm.qumranet.com/kvmwiki/TODO


KVM Pros

Leverages Linux scheduler, memory management, I/ONo scheduler involvement for I/OFull virtualisation: No changes to the guest necessary

Paravirt drivers available for better performance

Uses existing Linux security modelcan run VM as ordinary user

Uses existing management toolsPower managementGuest memory swappingReal-time scheduling, NUMALeverages Linux development momentum: all new drivers, {cpu, disk} schedulers, file systems, etc supported


Distro / Industry interest

libvirtManaging various guests under a hypervisorSupport for Xen, KVMAPIs between UI, middle layer and virtualisation backend

DistributionsDebianUbuntuRedHat ELSLES

QumranetDekstop Virtualisation


Release Philosophy

Development snapshots every 1-2 weeksRelease early and oftenFeatures introduces quicklyBugs fixed quicklyBugs added quicklyAllows developers and users to track and test the latest and greatest

Stable releases part of Linux 2.6.xWith bugfixes going into Linux 2.6.x.y


Journey

Linux 2.6.20 (4 Feb 2007): Initial releaseLinux 2.6.21 (25 Apr 2007): Stability, suspend/resumeLinux 2.6.22 (8 Jul 2007): Stable ABI

Old userspace, new kernelNew userspace, old kernel

Linux 2.6.23 (9 Oct 2007): SMP, performanceLinux 2.6.24 (24 Jan 2008): In-kernel APIC, preemptibility, virtioLinux 2.6.25 (16 Apr 2008): Guest swapping, paravirt_ops, balloon drvLinux 2.6.26 (soon): PowerPC, s390, IA64, NPT, EPT, more paravirt (mmu), ...


KVM is Developer-friendly

No need to reboot (usually)Netconsole, oprofile, all the tools workSmall codebaseFriendly community


Future

Consolidate various virtualisation solutions existing in the kernel

Started with move to virt/ from drivers/kvm/

More hardware features supportMore paravirtualisation supportImprove guest scalingBetter support for management layers like libvirtIntel Real Mode Emulation


Do Read

virt/*, arch/[x86|ia64|s390|powerpc]/kvm/*KvmForum2007 wiki page on http://[email protected]@lists.osdl.org

http://kvm.qumranet.com/kvmwiki/KvmForum2007

http://kvm.qumranet.com/

Thank You

The kvm virtualization way

Technology

pv linux guest

guest usingvirtio

guest swappin

host copyright

guest physical address

memory access guest

etcsupported qumranet

expected guest physical