Challenges in designing a type 1 hypervisor for ARM v7 VE Manish Jaggi [email protected] http://in.linkedin.com/in/manishjaggi
Nov 22, 2014
Challenges in designing a type 1 hypervisor for ARM v7 VE
Manish Jaggi [email protected]
http://in.linkedin.com/in/manishjaggi
Hypervisor Layering in Software Stack
• Type 1 HyperOne, Xen, PikeOS, OKL4, HyperV, Vmware ESX !• Type 2 KVM, VirtualBox, Virtual PC, Parallels, BlueStacks
Hardware
Hypervisor
UserSpace
Kernel
Hardware
OS Kernel
UserSpace
Kernel
Hypervisor
VM1 VM2
UserSpace
KernelUserSpace
Monolithic vs Microkernel Design
HyperOne Microkernel Design..
Hardware
Hypervisor
VM2 Driver VM
VM1 Root
InterVM IPC
Virtual Memory Scheduling Interrupt
Routing
Virtualization Stack
Device Drivers Device DriversDevice Drivers
• Hypervisor Privilege Mode (PL2) NS • Different set of CP15 registers for hypervisor • Stage 2 MMU and SMMU • GIC Virtual CPU I/f • Timer Virtualization (Virtual Timer Interrupts for guest / hypervisor)
ARM v7 Virtualization Extensions
Guest Stage 2 MMU Translation
GICv2
SMMU
Bare Minimal Hypervisor
• A hypervisor boots up like a normal smp os – MMU / GIC / UART / Timer drivers – Board specific / platform specific code – Needs a device tree – Has a console (shell)
• Hypervisor functions – Trap guests accesses – Emulate – Load/run/suspend/resume guests
vm_tvcpu_t
interrupts
Core_regs
Cp15_regs
VM State
vGIC State
virtual devices state
IPC Channels
Memory Assigned
Hypervisor keeps the state of each VM and its associated vcpus. !A VM has a machine definition and physical and virtual devices connected to it !State of virtual devices is kept with hypervisor like VGIC, console, network and other virtual devices. !VCPU has cpu_regs context, the cp15 regs and the descriptors of interrupts injected into the vcpu. !This state is saved and restored as and when the VM or vcpu is scheduled out / in. !There are two types of context saves, Shallow and deep.
• Hypervisor handles exceptions originated from hypervisor mode • Hypervisor also handles the exceptions originated from Guest
Mode, these are routed to hypervisor as hypervisor trap and hypervisor identifies the type of exception using a syndrome register.
• Virtual Devices’ Memory mapped address space access is trapped by hypervisor and the read write is emulated in hypervisor mode
• Guests can also issue hyper calls to the hypervisor. These are mainly used by para virtualised drivers.
• Privileged instructions such as WFI and WFE trap into hypervisor.
Hypervisor Exception Handling
• Hypervisor emulated GICD and injects interrupts in guest using GICH
• Hypervisor programs the Link registers
• Guest access the virtual CPU I/f in the GIC itself, no emulation is required
• There are 2 methods for hypervisor to b notified that guest has deactivated the interrupt • Maintenance Interrupt • UnderFlow Interrupt
Guest Interrupt HandlingGICD Driver
Guest
Hypervisor
SoCUART
VCPU I/F CPU I/F Hyp I/f Distributor
Level Interrupt
GIC
GICD Emulation
Trap
GICC Driver
• ARM provides a Physical timer register and a Virtual Timer register. Virtual Timer is an offset from physical timer
• Hypervisor Programs the CNTVOFF register before guest is resumed.
• ARM provides separate timer interrupt for hypervisor and guest.
• Timer interrupt is injected in same way as other interrupts
Timer Virtualization
Managing Virtual Device -‐ Virtio MMIO
• virtio is an abstraction layer over devices in a paravirtualized hypervisor.
• virtio mmio does not require a PCI device
15
• Debugging a guest can be done on hypervisor console • for dumping guest state on console • for setting software breakpoints
• or gdb interface can be added in hypervisor to debug a guest
Guest Debug Support
VM Management Summary
• Context Save and Restore • Booting and Loading • Scheduling • VCPU Management • Remote Guest Image loading
18
Remote Device Management
A typical use case of Virtual Machines on a device is an enterprise Virtual Machine to provide a secure and separate workspace environment for employees. !Basic management operations include creating/deleting, resuming/suspending and powering on/off of a Virtual Machine
Questions