Evolution of Graphics Hardware: 40 years since Sutherland’s HMD Henry FUCHS University of North Carolina at Chapel Hill Graphics Hardware 2008 Sarajevo.

Evolution of Graphics Hardware:40 years since Sutherland’s HMD

Henry FUCHS University of North Carolina at Chapel Hill

Graphics Hardware 2008 Sarajevo

GH2008: Evolution of Graphics Hardware (c) 2008 Henry Fuchs UNC Chapel Hill 22

AcknowledgementAssistance with preparation of this talk

•Hideyuki Tamura, Canon

•Greg Welch, UNC

•Mark Mine, Disney

•Steve Molnar, NVIDIA

•Adam Lake, Intel


Evolution of Graphics H/W:

from what point ?•Ivan Sutherland’s 1963 PhD Sketchpad

• Beginning of graphics, but no new hardware

•Sutherland’s 1965 “Ultimate Display” idea

• A vision, but no specific HW specified

•Sutherland’s 1968 3D HMD System

• Complete working system, needed all specialized HW


Ivan E. Sutherland

A Head-Mounted Three-Dimensional Display, 1968 Fall Joint

Computer Conference•Implemented all the components of a real-time interactive 3D graphics system:

•Display device: stereo, optical see-through

•Image generation: Real-time 3D perspective

•Head tracking: Real-time 6 DOF

•3D model creation

•Hand-tracking & interaction: Real-time 3 DOF (’69 ?)


Procedure for Examining Evolution of Graphics HW

•For each subsystem: display, image generation, tracking..

• 1968: what was implemented, state of the art

•~1988: sample state of the art

• 2008: current example subsystem

•Assessment: •The dream in 1968/1970

•Grade for where we are now: A, B, C, D


Sutherland HMD: Overview


Images from Sutherland’s ’68 HMD


Display device on ’68 HMD

•Optical see-through head-mounted display


HMD ca. 1985: VPL head-mount

•Closed display, not see-through

•LCD from SONY pocket TV

•wide angle optics (LEEP)•(no distortion correction)


HMD ca. 1995: video see-through HMD (UNC)

•Need for video, rather than optical, see-through


Current HM Displays•closed, total VR

•Fakespace wide5

•Video see-through: •Canon / Mixed Reality Labs VH2002

•InnerOptic (based on UNC design)


Assessment on HM Displays

•The dream at 1968 / 1970: Displays built in to our ordinary eyeglasses; see virtual objects throughout the eyeglass lenses (not a ‘virtual inset’)

•Grade for where we are now: C•significant progress

•not even close to adequate


Image generation:Sutherland’s 1968 system•Real time

•Line drawing

•No hidden line elimination

•Heroic work: everything built out of gate-level chips•3D transforms

•Clipping divider

•Graphics pipeline


Image generation: ca 1988

•Real-time full-color raster image generation

•Rapid, consistent progress throughout the 1980s and 1990s•100s of polygons /sec in 1980

•Million polygons / sec early 1990s- UNC Pixel-Planes 5 at siggraph 1991: 2M polygons/sec


Image generation now

•Sophisticated shading effects and millions of polygons /sec

•Commodity graphics chips & boards

•Rapid, continuing progress

•Image generation processors becoming ubiquitous –migrating even to mobile devices


Image Generation Assessment•Dream of 1968/1970: Realistic image generation

•Sutherland 1965 Ultimate display: can’t distinguish between virtual and real objects

•Grade for where we are now: • A+

•Are we finished?

•No! illuminate globally; integrate physics sim. with graphics; integrate image capture with image display,..

•Feels great for the field to be doing so well.


Head tracking: Sutherland’s 1968 system

•Two tracking systems implemented•ultrasound tracking with multiple emitters and receivers

•mechanical tracker


Head tracking (ultrasonic)Sutherland’s 1968 system

•Ultrasonic tracking •3 transmitters on head-mount

•4 receivers hanging from ceiling

•Measure phase changes

•Problem with ambiguity of number of cycles of u/s signals

•Problem exacerbated by heating system


Head tracking (mechanical)Sutherland’s 1968 system•Vertical pivot in ceiling

•Universal joint on top

•Universal joint on bottom

•Shaft slides in and out

•“Sword of Damocles”•Heavy and uncomfortable

•Works


Head Tracking ~1988: Polhemus Magnetic Tracking

•Developed for head tracking in cockpit

•Very close range

•Not for walking across a room

•Severe warping of space with metal, with other magnetic fields. Can be reduced with calibration


Large area trackers: ~ 1991 UNC

•10x12 ft space demonstrated at Siggraph 1991

•Multiple optical sensors on head, LEDs in ceiling tiles•Lateral effect photo-diodes, much faster than imagers

•One LED lighted at a time

•Predict most-useful LED to light each time slot


Head tracking Now

•Multiple good solutions

•Optical

•Magnetic

•Inertial / optical combination

•Limited to instrumented area


Head tracking assessment•Dream of 1968 / 1970: Go anywhere, unencumbered

•Grade for where we are now:

• B•Choice of solutions on the market if willing to live with restrictions

•Degraded performance if lose line of sight contact (optical trackers) or near ferrous/metal objects (magnetic trackers)

•Can’t go outside of instrumented area, nor outdoors.

•Steady progress


Interaction / Hand tracking in Sutherland’s HMD

•Mechanical design with three reels of fishing line mounted from ceiling

•All lines connected to top of (camera) hand-grip

•Amount of line reeled out determines 3D location of hand-grip

•Problem of interference between head and hand trackers


Interaction / Hand tracking ca.1988

•Polhemus magnetic tracker (time-shared with a target for head-tracker)

•VPL DataGlove sometimes added to hand-tracker

•Severe warping of tracked space if user moves more than a few steps


Interaction / Hand tracking now•Multiple technologies available

•Accuracy & speed may not be adequate for certain applications•UNC AR assistance for surgical needle guidance

•Track ultrasound transducer

•Track needle


Assessment of Hand tracking•Dream in 1968/ 1970: unencumbered tracking of hand

•Grade for where we are now:

• Incomplete

• Haven’t had sufficient number of real applications to judge --- rest of system hasn’t been ready

•using Fred Brooks’ denition of “What’s real about Virtual Reality?”

•Someone other than develops pay to use the system


3D model creation•Sutherland’s 1968 system: trivially simple


3D model creation ca. 1988

•Manual model creation increasingly expensive / time-consuming with increasing capability of image generation systems

•Feasible if 3D model needed to be created as part of the application itself: CAD/CAM, selected medical applications

•Automatic scanning for selected objects: Cyberware 3D scanners


3D Model Creation Now

•Growing list of commercial and research capabilities

•Computer vision: multiple images / video to 3D world models

•Laser scanners

•Motion capture systems


Assessment of Model Creation•Dream in 1968 / 1970:

•Sutherland: Building model may be as time-consuming as building the real thing

•Others: There’s got to be an automatic way

•Grade for where we are now

• C+•There are many tools and systems, but often still takes much tedious, manual effort

•Compare how much more work to digitizing this room and its contents than to rendering it


Assessment Summary•Display:

• perennially, next model will be really good C

•Image generation:

• unalloyed triumph! (but don’t let it go to your head) A+

•Head tracking:

• effective but in limited, prepared spaces B

•Interaction / Hand tracking:

• awaiting more real applications Incomplete

•Model generation:

• frustratingly time-consuming C+


Opportunities: Look at larger picture than a single module•Head-mounted display design combined with

image generation•Fakespace Wide5: hardware geometric distortion

correction

•Mark Mine (Disney) implementing more sophisticated version in GPU

•Image generation combined with head-tracking

•Model creation combined with image generation

•System problem spread across multiple modules:•latency


Sutherland’s vision of the HMD as the penultimate display still inspires


In conclusion:

Don’t constrain your dreams;there is so much cool graphics HW to invent•Stereo and autostereo displays

•Multi-projector / camera systems

•Vision capture for image-based rendering

•Haptics in the small and large

•Cell phones w/graphics & vision: magic lenses in the real world

•Telepresence systems (could do an entire hour on this)

•Finally– Alan Kay The best way to predict the future is to invent it THE END

35

Evolution of Graphics Hardware: 40 years since Sutherland’s HMD Henry FUCHS University of North Carolina at Chapel Hill Graphics Hardware 2008 Sarajevo.

Documents

chapel hill graphics

henry fuchs unc chapel

evolution of graphics

hmd slide

unc design slide

d graphics system

unc mark

hmd unc need