Workshop on Parallel Visualization and Graphics Chromium Mike Houston, Stanford University and The Chromium Community.

Workshop on Parallel Visualization and Graphics

Workshop on Parallel Visualization and Graphics

Chromium

Mike Houston, Stanford University

and

The Chromium Community

2Workshop on Parallel Visualization and Graphics

How Chromium works

Replaces system’s OpenGL driver• Industry standard API• Support existing unmodified applications

Manipulates streams of API commands• Alter/inject/discard commands and parameters• Route commands over a network• Render commands using graphics hardware• State tracking

Allows parallel applications to issue OpenGL• Constrain ordering between multiple streams


Graphics Stream Processing

Treat OpenGL calls as a stream of commands Form a DAG of stream transformation nodes

• Nodes are computers in a cluster• Edges are OpenGL API communication

Each node has a serialization stage and a transformation stage


Stream Serialization

Convert multiple streams into a single stream Efficiently context-switch between streams Constrain ordering using Parallel OpenGL extensions

[Igehy98] Two kinds of serializers:

• Network server:

• Application:• Unmodified serial application• Custom parallel application

S

AOpenGL


Stream Transformation

Serialized stream is dispatched to “Stream Processing Units” (SPUs)

Each SPU is a shared library• Exports a (partial) OpenGL interface

Each node loads a chain of SPUs at run time SPUs are generic and interchangeable


SPU Chains

SPUs are loaded as parts of linear chains

Common usage: intercept a few OpenGL calls, pass all others to downstream SPU

Useful for simple state changes, such as “wireframe” drawing


Output Scalability (Sort-First)

Larger displays with unmodified applications Other possibilities: broadcast, ring network

App

Server

Server

Server

...

Display

Display

Display

...


Example: Sort-First

App

Tilesort ...

Server

Server

Server

Render

Render

Render


Input Scalability (Sort-Last)

Parallel geometry extraction Parallel data submission

App

App

App

...DisplayServer


Example: Sort-Last

Application runs directly on graphics hardware Same application can use sort-last or sort-first

...

Application

Application

Application

Readback

Readback

Readback

Send

Send

Send

Server

Render


SPU Inheritance

The Readback and Render SPUs are related• Readback renders everything except SwapBuffers

Readback inherits from the Render SPU• Override parent’s implementation of SwapBuffers• All OpenGL calls considered “virtual”


Readback’s SwapBuffers

Easily extended to include depth composite All other functions inherited from Render SPU

void RB_SwapBuffers(void){ self.ReadPixels( 0, 0, w, h, ... ); child.Clear( GL_COLOR_BUFFER_BIT ); child.SemaphorePCR( READBACK_SEMAPHORE ); child.RasterPos2i( tileX, tileY ); child.DrawPixels( w, h, ... ); child.SemaphoreVCR( READBACK_SEMAPHORE ); child.SwapBuffers( );}


More Complicated Example: Hybrid

App

App

App

...Tilesort

Tilesort

Tilesort

...

Server

Server

Server

Readback

Readback

Readback

Send

Send

Send

Server

Render


Networks Supported

TCP/UDP Myrinet Quadrics Infiniband (coming soon)


New Things to Chromium

Extensions DMX Support Display list management (DLM) VNC Support CRUT Dale’s talk


Extensions

GL_ARB_fragment_program GL_ARB_vertex_program GL_NV_fragment_program GL_NV_vertex_program GL_NV_texture_rectangle GL_EXT_shadow_funcs GL_EXT_texture_rectangle GL_IBM_raster_pos_clip


DMX Support

DMX• Distributed Multi-headed X• Single X session across multiple-displays

OpenGL through Chromium• Chromium “DMX aware” • Moving/resizing = retiling• M to N rendering


DMX In Action


Display List Management

Display List Manager (DLM)• State tracking is really tricky

• Replay state calls on client• Call list on servers

• Bounding Box tracking of display list Future optimizations

• Avoid broadcasting data in display list• Send calls once per server as needed


VNC

X forwarding• Forwards GLX calls to client

DRI bypasses X• Can’t get pixel data

OpenGL apps load Chromium• Render on local host• Readback pixel data• Send to user’s display


What are people doing with Chromium?


Dynamic Screen Calibration


Quake3 Arena

Niederauer, et al.


Viewed in a new way

Niederauer, et al.


Architectural Analysis

Intercept geometry Determine floor positions Change to orthographic view Insert clip planes at the ceilings Split floors apart Multi-pass rendering

“Non-Invasive Interactive Visualization of Dynamic Architectural Environments”

Christopher Niederauer, Mike Houston, Maneesh Agrawala, Greg Humphreys

ACM SIGGRAPH 2003 Symposium on Interactive 3D Graphics


Batch Scheduler Integration

Offline rendering to a webpage Use massive compute resources Rendering with Vis cluster Integrate support with RMS

Pittsburg Supercomputer Center


Summary

• 750 Compute Nodes• 3000 EV68 processors

• 6 Tf (peak, est >4Tf on LSMS)• 3. TB memory• 27 TB local disk• Multi-rail fat-tree network• Redundant monitor/ctrl• WAN/LAN accessible• Parallel visualization

• File servers: 30TB, ~32 GB/s• Mass store, ~1 TB/hr

WAN/LAN

Switchedethernet

Quadrics

Control

Compute Nodes

File Servers

VizMass Store

bufferArchive

Interactive/home

Terascale Computing System



Example

qsub –l rmsnodes=3:12,other=visnodes=5 Job waits until 3 nodes (12 cpus) become

available AND 5 vis nodes are available When resources available, job runs Visit vis web page for rendering



What coming in the next year?


General Improvements

Continue to track OpenGL changes• Add extensions

Optimizations• Display list management• Tilesort

Software Compositors


PICA Support

Parallel Image Compositing API (PICA)• API for hardware and software compositing• Will be supported by most hardware compositors

Chromium support• Hooks almost complete• Need software compositors

• Readback (N to 1)• Binary-swap• SLIC

• Need info from hardware folks


“Vis as a service”

Better integration with schedulers• Reservation systems• Compute/Render/Display

Distributed event model (CRUT) Compression

• Geometry data• Pixel data

Encryption


Look at how much was done last year!

4 releases Constant bug fixes Constant improvements Constant optimizations

Chromium is used in the real world!

Chromium is supported by a large community!


Go get it!

http://chromium.sourceforge.net


Acknowlegements

The Chromium community Greg Humphreys Brian Paul Joel Welling Alan Hourihane DOE!!!

Workshop on Parallel Visualization and Graphics Chromium Mike Houston, Stanford University and The Chromium Community.

Documents

parallel visualization

opengl slide

parallel applications

graphics stream processing

server render slide

interchangeable slide

virtual slide

display server slide