Transcript
ARMvisorPeter Chang
This side is licensed under CC-BY-NC-SA姓名標示─非商業性─相同方式分享
(http://creativecommons.org/licenses/by-nc-sa/3.0/tw/legalcode)
Who am I?
Who am I?
• Peter Chang
Who am I?
• Peter Chang
• One of the current developers of ARMvisor
Who am I?
• Peter Chang
• One of the current developers of ARMvisor
• http://tw.linkedin.com/in/peterchangtw
What is ARMvisor?
ARM
ARM
ARM
What is KVM?
What is KVM?
• A.k.a. Kernel-based Virtual Machine
What is KVM?
• A.k.a. Kernel-based Virtual Machine
• Type-II Virtual Machine Monitor
What is KVM?
• A.k.a. Kernel-based Virtual Machine
• Type-II Virtual Machine Monitor
• A module of Linux kernel
What is KVM?
What is KVM?
• Officially support x86/x64, PowerPC, S390
What is KVM?
• Officially support x86/x64, PowerPC, S390
• No official support for ARM architecture
What is ARMvisor?
What is ARMvisor?
• ARM架構上的KVM
What is ARMvisor?
• ARM架構上的KVM
• Para-virtualization
What is ARMvisor?
• ARM架構上的KVM
• Para-virtualization
• Trap & Emulation
What is ARMvisor?
• ARM架構上的KVM
• Para-virtualization
• Trap & Emulation
• Dynamic Memory Allocation
What is ARMvisor?
• ARM架構上的KVM
• Para-virtualization
• Trap & Emulation
• Dynamic Memory Allocation
• virtio & IRQchip-in-kernel
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisorDriver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
2012
2011
2010
2009
2012
2011
2010
2009
2012
2011
2010
2009
Starting Point
2012
2011
2010
2009
Starting Point
2012
2011
2010
2009
Starting Point
ARMvisor Prototype
2012
2011
2010
2009
Starting Point
ARMvisor Prototype
2012
2011
2010
2009
Starting Point
ARMvisor Prototype
CPU Opt & Mem Opt
2012
2011
2010
2009
Starting Point
ARMvisor Prototype
CPU Opt & Mem Opt
2012
2011
2010
2009
Starting Point
ARMvisor Prototype
CPU Opt & Mem Opt
I/O Opt
Supported Hardware
ARM Realview-eb
ARM11
ARMv6 ISA
Supported Hardware
TI BeagleBoard
Cortex-A8
ARMv7 ISA
Supported Software
• Patched Host OS:
• Linaro Linux 2.6.38
• Host Root Filesystem
• Ubuntu/Debian RFS (CLI or GUI)
• QEMU 0.14
Supported Software
• Patched Guest OS:
• Linux 2.6.35 (running on ARMv6 ISA)
• Guest Root Filesystem:
• Ubuntu/Debian RFS (CLI or GUI)
System model of ARMvisor
CPU virtualization
CPU virtualization
• ARM is non-virtualizable CPU
• Patch guest OS
• “Trap and emulation”
ARMv6 ISA1.Branch instructions2.Data-processing instructions3.Multiply instructions4.Parallel addition and subtraction instructions5.Extend instructions6.Miscellaneous arithmetic instructions7.Other miscellaneous instructions8.Status register access instructions9.Load and store instructions10.Load and Store Multiple instructions11.Semaphore instructions12.Exception-generating instructions13.Coprocessor instructions
ARMv6 ISA1.Branch instructions2.Data-processing instructions3.Multiply instructions4.Parallel addition and subtraction instructions5.Extend instructions6.Miscellaneous arithmetic instructions7.Other miscellaneous instructions8.Status register access instructions9.Load and store instructions10.Load and Store Multiple instructions11.Semaphore instructions12.Exception-generating instructions13.Coprocessor instructions
Sensitive Instructions ?
ARMv6 ISA1.Branch instructions2.Data-processing instructions3.Multiply instructions4.Parallel addition and subtraction instructions5.Extend instructions6.Miscellaneous arithmetic instructions7.Other miscellaneous instructions8.Status register access instructions9.Load and store instructions10.Load and Store Multiple instructions11.Semaphore instructions12.Exception-generating instructions13.Coprocessor instructions
Data-processing instructions S-BIT: MOVS, ...
Status register access instructions MRS, MSR, CPS, SETEND
Load and store instructions T-BIT: LDRT, STRT, ...
Load and Store Multiple instructions LDM(2), LDM(3), STM(2)
Exception-generating instructions SWI, BKPT
Coprocessor instructions MCR, MRC, MCRR, ...
Sensitive Instructions
Observation from Guest Linux Code
• MOVS• (MRS, MSR, CPS) • (LDRBT, LDRT, STRBT, STRT)• (LDM(2), LDM(3), STM(2))• SWI• (MCR, MRC, MCRR)
15 sensitive instructions used in the guest linux code
“Trap & emulation”
User%space%%
Kernel%space�
Guest%OS�
VMM�
trap�
How to “trap”?
…mov r0, r0add sp, spmovs pc, lr…
How to “trap”?
…mov r0, r0add sp, spvirt_svc_movs “movs pc, lr”…
How to “trap”?
…mov r0, r0add sp, spvirt_svc_movs “movs pc, lr”…
How to “trap”?
…mov r0, r0add sp, spvirt_svc_movs “movs pc, lr”…
.macro virt_svc_movs, instSWI 0x190\inst.endm
How to “emulate”?
oxffff0000
oxffff1000
Kernel Vector0xffff001c
oxffff0000
oxffff1000
Kernel Vector0xffff001c
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
KVM Trap Entry
KVM/Guest Context Switch
UnitHost Trap Handler
Instruction Emulation
Exception/Interrupt Emulation
MMU Emulation
QEMU I/OEmulation
KVM Trap Dispatcher
UND ABORT SWI IRQ/FIQ
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
1. VM initialization
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit GuestLightweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
Lightweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
7. Exit Guest
Lightweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
7. Exit Guest8. Return to QEMU
Lightweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
7. Exit Guest8. Return to QEMU
Lightweight trap
Heavyweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
7. Exit Guest8. Return to QEMU
9. Run VM
Lightweight trap
Heavyweight trap
Kernel space
KVM
User space
QEMU
Guest Mode
Guest OS
2. Return to QEMU
1. VM initialization
3. Run VM4. Enter Guest
5. Exit Guest
6. Enter Guest
7. Exit Guest8. Return to QEMU
9. Run VM
10. Enter Guest
Lightweight trap
Heavyweight trap
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
VCPU Register
FileSync
oxffff2000
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
Shadow Register FileVCPU
Register File
Sync
oxffff2000
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
Shadow Register File
mcr cpsr, r1
VCPU Register
FileSync
oxffff2000
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
Shadow Register File
mcr cpsr, r1
VCPU Register
FileSync
oxffff2000
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
Shadow Register FileRead/WriteInstructions
mcr cpsr, r1
VCPU Register
FileSync
oxffff2000
oxffff0000
oxffff1000
Kernel Vector0xffff001c
The KVM trapInterface
Shadow Register FileRead/WriteInstructions
mcr cpsr, r1
VCPU Register
FileSync
oxffff2000
Memory virtualization
Virtual Address
Physical Address
PTB
GVA
GPA
HVA
HPA
Guest PTBGVA
GPA
HVA
HPA
Guest PTBGVA
GPA
HVA
HPA
Guest PTBGVA
GPA
HVA
HPA
Host PTB
Guest PTBGVA
GPA
HVA
HPA
Host PTB
New SPTE !!!
PABT/DABT trap
PABT/DABT trap
guest page table walker
PABT/DABT trap
True Translation fault
guest page table walker
Guest permission checker
PABT/DABT trap
True Translation fault
guest page table walker
Guest permission checker
PABT/DABT trap
True Translation fault
True permission fault
guest page table walker
Guest permission checker
MMIO access checker
PABT/DABT trap
True Translation fault
True permission fault
guest page table walker
Guest permission checker
MMIO access checker
PABT/DABT trap
MMIO emulationTrue Translation fault
True permission fault
guest page table walker
Guest permission checker
Shadow page table
mapping
MMIO access checker
PABT/DABT trap
MMIO emulationTrue Translation fault
True permission fault
guest page table walker
Guest permission checker
Shadow page table
mapping
MMIO access checker
PABT/DABT trapHidden protection fault
MMIO emulationTrue Translation fault
True permission fault
guest page table walker
Guest permission checker
Shadow page table
mapping
MMIO access checker
PABT/DABT trapHidden protection fault
Hidden translation fault
MMIO emulationTrue Translation fault
True permission fault
guest page table walker
Guest permission checker
Shadow page table update
Shadow page table
mapping
MMIO access checker
PABT/DABT trapHidden protection fault
Hidden translation fault
MMIO emulationTrue Translation fault
True permission fault
guest page table walker
I/O virtualization
I/O virtualization
• Emulation by QEMU
• virtio
• IRQ chip in kernel
Emulate by QEMU
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
1
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
1
2
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
1
23
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
1
23
4
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
5
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
6
5
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
76
5
Hardware: ARM Cortex-A8
Host OS: Linux 2.6.38
ARMvisor
Driver
QEMU 0.14
Device
Driver
Guest OS: Linux 2.6.35
8
76
5
virtio
virtio
Vir,o AMBA Controller
Vir,o Driver Guest
Vring Transport
Vir,o AMBA Controller
Vir,o DeviceQEMU
irq_chip in kernel
Interrupt ControllerOpera,ng System
Deliver Interrupt
Get IRQ number
Ack IRQ number1
Mask IRQ2
End of IRQ3
Unmask IRQ4
irq_chip in kernel
ARMvisor�
GIC�
Device� Device� Device�
QEMU�
Guest�
IOCTL7
Deliver7IRQ7
Control7GIC7
irq_chip in kernel
GIC$in$Kernel�
Device� Device� Device�
QEMU�
Guest�
KVM�
Deliver$IRQ$
Control$GIC$
Future Works
Future Works
Future Works
• Support for ARM’s Virtualization extension
Future Works
• Support for ARM’s Virtualization extension
• Cortex-A15 and beyond
Future Works
• Support for ARM’s Virtualization extension
• Cortex-A15 and beyond
• SMP for host and guest
Future Works
• Support for ARM’s Virtualization extension
• Cortex-A15 and beyond
• SMP for host and guest
• AArch64 Support
Future Works
• Support for ARM’s Virtualization extension
• Cortex-A15 and beyond
• SMP for host and guest
• AArch64 Support
• ARMv8
By the way, ...
OpenSource
OpenSource
• We HAVE opened source in late August, 2012.
OpenSource
• We HAVE opened source in late August, 2012.
• GNU GPLv2
OpenSource
• We HAVE opened source in late August, 2012.
• GNU GPLv2
• Source code of Host and Guest OS
OpenSource
• We HAVE opened source in late August, 2012.
• GNU GPLv2
• Source code of Host and Guest OS
• https://github.com/SSLab-NTHU
Q & A
top related