Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.

8/8/2019 Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.

1/56

Study of secure isolation of virtual machines

and

their exposure to hosts in a virtual

environment.

Gavin Fitzpatrick

School of Computing

Dublin City University

Dublin, Ireland


2/56

Virtualization Concepts

Type 1 Hypervisor used Type 2 Hypervisor used

Testing tools

Experiments

Results

Conclusions


3/56

Virtualization Concepts

x86 architectures are designed based on 4 rings of privilege:

Ring 3: executes user mode - has no direct access to the underling hardware

Ring 2: not used by modern operating systems.

Ring 1: not used by modern operating systems.

Ring 0: has full access to underlying hardware within the host system

Popek & Goldberg define an x86 Virtual Machine Monitor(VMM) with the

following characteristics:

Fidelity:V

MM must provide computing environment identical to a physical machine Performance: Programs should only have minimum performance impact when using a

VMM

Safety: The VMM must have complete control of the system resources

Ref: Popek & Goldberg Formal requirements for Virtualizable 3rd Generation Architectures


4/56

Virtualization Isolation

As discussed in *Virtual Doppelganger paper,

Isolation within virtualization can be defined

under two different dimensions:

Resource Isolation

Namespace Isolation

*Ref: http://www.cs.princeton.edu/~mef/research/paenevirtualization.pdf


5/56

Namespace Isolation

Namespace Isolation:

States how a VMM limits access to its file-system,processes, memory addresses, user ids etc.

2 aspect:

1. Configuration Independence: File names ofone VM do not conflict with that of anotherV

M. All hypervisors tested were unable to use the

same name for vms or their associated config/virtual disk files (vmdk, vdi, vhd)


6/56

Namespace Isolation

2. Security:

One VM cannot modify data belonging to

another VM stored in the same host.

Within modern enterprise level environments

VMs are stored on iSCSI or Fibre-channel

networks which are inaccessible from VMs


7/56

Resource Isolation

Refers to a VMMs ability to isolate resource

consumption of one VM from that of another

VM using appropriate algorithms:

This Presentation looks at how resource

isolation is affected by VMs misbehaving.


8/56

Hypervisor (VMM)

Type 1: ESXi, XEN Server, Hyper-V

Type 2: VMWare Workstation, Oracle VirtualBox

Containers: Virtual code that runs as an application,

Allows multiple encapsulated isolated instances which point to the underlying O.Son which its executed

*diagrams from: http://www-01.ibm.com/redbooks/community/display/REDP4480/Virtualization+Strategies+Architectural+Overview


9/56

ESXi


10/56

XENServer


11/56

Hyper-V


12/56

KVM


13/56

Virtualbox


14/56

Workstation


15/56

Testing tools

Commercial benchmarking tools includeVMark , Passmark and

All tests executed on VM4 (Ubuntu Guest)

Ramspeed

Systester

Geekbench

FIO

Ping testing ( Look at skipping?!)


16/56

Ramspeed

Used to test memory performance with the following operations:

Copy (A=B)

Scale (A=m*B)

Add (A = B+C)

Triad (A=m*B+C)

2 tests are performed for Integers and Floating Point Numbers

10 rounds are performed for each test and results are averaged


17/56

Systester

Used to benchmark CPU performance by

calculating 512,000 digits of Pi using 2 algorithms:

Borwein Quadradic Covergance: Runs for 5 rounds

Gausse-Legendre: Runs for 10 rounds


18/56

Geekbench

Propreitary benchmarking tool used for memory & cpuperformance

Scores tested on following factors:

1. Integer Calculations (Blowfish, Text Compress/Decompress)

2. Floating Point calculations (Primality test, Dot Product)3. Memory operations (Read/Write Sequential, Stdlib Copy/Write)

4. Stream operations similar to Ramspeed( Copy, Scale, Add, Triad (similar to ramspeed tests)


19/56

FIO

I/O benchmark tool used to test disk subsystemwithin Linux O.S with libaio library

Test performed:

Random write: 10 x 32mb files written Max average bandwidth recorded over 10

reads/writes


20/56

Ping testing

Tests Network I/O within each hypervisor, pingtests run from VM4 within the host to

Gateway

Host IP (Physical IP for host) VM2 (Win2003 server DoS victim during Exp4a/b)


21/56

Testing Script


22/56

Experiments

Testing Environment

Crashme O.S stress test

Fuzz Application stress test

Forkbomb Memory stress test

DoS I/O stress test


23/56


24/56

Exp1 - Crashme


25/56

Exp1 VM1 CPU/MEM activity


26/56

Crashme - Observations

Exp1:

CPU: 100% usage

MEM usage: 75%

PRNGs used: Mersenne Twist (MT) Common PRNG tool

VNSQ (Variation of Middle Sq. Method) Take any No. Sq it, and take middle digits

Rand() uses c++ library

Virtualbox- in non VT-x mode: Running Exp1 on Virtualbox in non VT-x mode causes the O.S to hang using (MT) method

Both VNSQ and Rand() cause O.S to restart


27/56

Exp2 - Fuzz

Exp2:

CPU: 88% + usage

MEM: 30%+ usage


28/56

Exp3 - Forkbomb


29/56

Exp3 - Forkbomb

Forkbomb code Linux (VM3): defined in a bash script as:

Forkbomb code Windows (VM1,VM2): defined in batch as :%0|%0

REF: http://www.cyberciti.biz/faq/understanding-bash-fork-bomb/


30/56

Exp3a-c Host MEM activity


31/56

Exp4 - DoS

Exp4a:

CPU: 100% usage

NIC transmit/recieve: 10,000KBps / 5,500KBps

Exp4b:

CPU: 25% usage

NIC transmit/ recieve: 0 / 13,500KBps


32/56

Results

Illustrated on a test by test basis for all

experiments for the following:

ESXi XEN

Hyper-V

Workstation Virtualbox / Virtualbox nonVTx


33/56

ESXi - Memory

Ramspeed:

Y-axis in MB persec(higher score=better result)

Geekbench:

Y-axis is score (higherscore=better result)


34/56

ESXi - CPU

Systester:

Y-axis in time (lower time better result)

Geekbench:

Y-axis is score (higherscore=better result)


35/56

ESXi HDD/LAN

0

200

400

600

800

1000

1200

ESX KB/s

Avg Write per Exp

KB/s

0

0.05

0.1

0.15

0.20.25

0.3

0.35

0.4

0.45

ESX

Ping Host - Secs

0

0.2

0.4

0.6

0.8

1

1.2

1.4

ESX

Ping Gateway -

Secs

0

0.5

1

1.5

22.5

3

3.5

4

4.5

5

ESX

Ping VM - Secs


36/56

ESXi - Summary

Memory: Geekbench: 2.2% better than average

Ramspeed: 2.5% better than average

CPU: 5% better than average

Disk: 18% below average (especially in Exp3c)

Network:

Host: 55% above average VM: 22% above average

GW: 19% above average


37/56

XEN-Memory


38/56

XEN-CPU


39/56

XEN-Disk/Network

0

200

400

600

800

1000

1200

1400

XenServer

Avg Write per Exp

KB/s

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

XenServerPing Host - Secs

0

1

2

3

4

5

6

XenServer

Ping VM - Secs

0

0.2

0.4

0.6

0.8

1

1.2

1.4

XenServer

Ping Gateway -

Secs


40/56

XEN - Summary

Memory: Geekbench: Follows average apart from Exp3c

Ramspeed: 4.5% below average (Exp3c, Exp4b majorfactors)

CPU: 3% better than average

Disk: 41% greater performance than average

Network: Host: 20.3% above average (Exp4a performs badly)

VM: 31% below average

GW: 16.4% above average


41/56

Hyper-V - Memory


42/56

Hyper-V - CPU


43/56

Hyper-V HDD/LAN

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Hyper-V

Ping Host - Secs0

1

2

3

4

5

6

Hyper-V

Ping VM - Secs

0

0.5

1

1.5

2

2.5

3

3.5

Hyper-V

Ping Gateway - Secs


44/56

Hyper-V Summary

Memory: Geekbench: Exp1,2,3a score below average, Exp3b-4b score

above average

Ramspeed: 3.4% below average (Exp3c-4b main cause)

CPU: 2.5% below average (resulting from Borwein tests)

Gauss test follows average

Disk: 18% above average (Exp3b, 3c show major loss in performance)

Network: Host: 81% below average

VM: 31% below average

GW: 4.5% below average


45/56

Workstation - MEM


46/56

Workstation - CPU


47/56

Workstation HDD/LAN

0

200

400

600

800

1000

1200

WorkstationKB/s

Avg Write per Exp KB/s

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Workstation

Ping Host - Secs

0

1

2

3

4

5

6

Workstation

Ping VM - Secs

0

0.5

1

1.5

2

2.5

3

3.5

Hyper-V

Ping Gateway - Secs


48/56

Workstation - Summary

Memory: Geekbench: 1.1% below average (Exp3c-4b main cause)

Ramspeed: Integer tests 3.3% abover average, FloatingPoint tests 6% above average

CPU: 1.2 % below score across all experiments

Disk: 19% below average keeping in line with average trend

Network: Host: 5.1% better than average

VM: 11.4% better than average

GW: 10% better than average


49/56

Virtualbox - Memory


50/56

Virtualbox - CPU


51/56

Virtualbox HDD/LAN

0

200

400

600

800

1000

1200

1400

1600

Virtualbox KB/s

Avg Write per Exp KB/s

0

0.1

0.2

0.3

0.4

0.5

0.6

Virtualbox

Ping Host - Secs

0

1

2

3

4

5

6

Virtualbox

Ping VM - Secs

0

0.2

0.4

0.6

0.8

1

1.2

1.41.6

Virtualbox

Ping Gateway - Secs


52/56

Virtualbox - Summary

Memory: Geekbench: 1.1% below average (Exp3c-4b main cause)

Ramspeed: Integer tests 3.3% above average, FloatingPoint tests 6% above average

CPU: 1.2 % below score across all experiments

Disk: 19% below average keeping in line with average trend

Network: Host: 15% below average across all experiments

VM: 5% below average across all experiments

GW: 6.1% below average across all experiments


53/56

Conclusions

Type 1 Baremetal (ESXi)

Outperforms all hypervisors on:

Network (utilizes NIOC) CPU/MEM ( CPU Scheduler / Shadow Page tables)

Performs poorly for:

Disk access (SIOC doesnt enforce isolation)


54/56

Conclusions

Type 1 Para Virtualization (XEN & Hyper-V)

Disk I/O performs well on both platforms Due to ability of Guests to utilize the Domain0 hardware

device drivers

Network I/O per poorly XEN & Hyper-V both perform poorly for Network I/O

isolation

Mixed CPU/MEM results: XEN: Average Memory performance, better than average

CPU performance

Hyper-V Poor CPU/MEM performance


55/56

Conclusions

Type 2 Hosted (Virtualbox & Workstation)

Disk I/O performs poorly on both platforms

Network I/O mixed results: Virtualbox performs poorly across Network tests

Workstation performs well across Network tests

CPU performs poorly:

Both platforms report lower than average CPU results, this is due toHost CPU treating each Guest as a separate process, which has the

same CPU time slice allocation as other Host Context Ring 3 processes Memory performs well:

Both platforms perform well for memory access, VMM allocatephysical memory directly to each Guest, host in unaware of this.


56/56

Practicum Presentation: Study of resource isolation within virtual enviroments of misbehaving virtual machines.

Documents