The Role of Virtualization in Exascale Production Systems

The Role of Virtualization in Exascale Production Systems

Jack LangeAssistant Professor

University of Pittsburgh

Where are exascale OSes headed?• Linux is accepted on Petascale

– More than 80% of Top500– Significant push to run Linux on production systems

• Pros– Complex systems are easier to manage– Existing codebase– Familiar and portable environment

• Cons– Not as responsive to application requirements– Exascale is not a primary design goal

• Linux is great for Ops, but only OK for Apps

Lightweight Kernels (LWKs)• What do Apps need?

– Simple memory management with little to no overhead– Low noise characteristics– Ability to perform large “bulk” I/O– …

• Traditional strengths of Lightweight Kernel architectures– Application driven resource management

• Lightweight kernels still have utility– But probably not as the primary system OS– App selectable runtime environment

• Challenge– Can we provide a lightweight environment on a heavyweight OS?

Virtualization on Linux• Bypass Linux management layers

• Palacios selectively takes over resource management– Memory, devices, CPUs– Repurpose existing mechanisms

• Allocate large chunks of resources and manage them internally

• Kernel module– Does not require kernel modifications – Implements lightweight interface– Compatible with CNL (~2.6.32)

Available in Palacios 1.3 (Nov. 2011)

Virtualized Dual Software Stack

Hardware

Palacios VMM

Lightweight Kernel

ManagementProcesses

+System Daemons

Linux Module Interface

Linux derived Compute Node

OS

HPC Application

Palacios ResourceManagers

• LWKs manage large resource allocations from VMM• Apps can choose which allocators/management layers to deploy on• LWKs can be co-designed with Apps and deployed on production systems

Performance isolationMemory Performance (GB/s)

Noise Comparison between native Linux and virtual LWK

Thank you• Jack Lange

– [email protected]– http://www.cs.pitt.edu/~jacklange

• V3Vee Project – http://www.v3vee.org