COSC 426: Augmented Reality Mark Billinghurst [email protected] July 18 th 2012 Lecture 2: AR Technology
426 lecture2: AR Technology
May 18, 2015
The second lecture for the COSC 426 postgraduate course on Augmented Rea
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
COSC 426: Augmented Reality
Mark Billinghurst
July 18th 2012
Lecture 2: AR Technology
Key Points from Lecture 1
Augmented Reality Definition Defining Characteristics [Azuma 97]
Combines Real and Virtual Images - Both can be seen at the same time
Interactive in real-time - Virtual content can be interacted with
Registered in 3D - Virtual objects appear fixed in space
What is not Augmented Reality?
Location-based services Barcode detection (QR-codes) Augmenting still images Special effects in movies … … but they can be combined with AR!
Milgram’s Reality-Virtuality Continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real Environment
Augmented Reality (AR)
Augmented Virtuality (AV)
Virtual Environment
Metaverse
AR History Summary 1960’s – 80’s: Early Experimentation 1980’s – 90’s: Basic Research
Tracking, displays
1995 – 2005: Tools/Applications Interaction, usability, theory
2005 - : Commercial Applications Games, Medical, Industry
Applications
Medicine Manufacturing Information overlay Architecture Museum Marketing Gaming
AR Technology
“The product is no longer the basis of value. The
experience is.”
Venkat Ramaswamy The Future of Competition.
experiences
services
products
components
Valu
e
Sony CSL © 2004
Gilmore + Pine: Experience Economy
Function
Emotion
experiences
applications
tools
components
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability
experiences
applications
tools
components
Sony CSL © 2004
Building Compelling AR Experiences
Display, Tracking
AR Technology Key Technologies
Display Tracking Input Processing
Display
Processing
Input
Tracking
AR Displays
AR Displays
e.g. window reflections
Virtual Images seen off windows
e.g. Reach-In
Projection CRT Display using beamsplitter
Not Head-Mounted
e.g. Shared Space Magic Book
Liquid Crystal Displays LCDs
Head-Mounted Display (HMD)
Primarily Indoor Environments
e.g. WLVA and IVRD
Cathode Ray Tube (CRT) or Virtual Retinal Display (VRD)
Many Military Applications & Assistive Technologies
Head-Mounted Display (HMD)
e.g. Head-Up Display (HUD)
Projection Display Navigational Aids in Cars
Military Airborne Applications
Not Head Mounted (e.g. vehicle mounted)
Primarily Outdoor (Daylight) Environments
AR Visual Displays
Head Mounted Displays
Head Mounted Displays (HMD) - Display and Optics mounted on Head - May or may not fully occlude real world - Provide full-color images - Considerations
• Cumbersome to wear • Brightness • Low power consumption • Resolution limited • Cost is high?
Key Properties of HMD Field of View
Human eye 95 degrees horizontal, 60/70 degrees vertical
Resolution > 320x240 pixel
Refresh Rate Focus
Fixed/manual
Power Size
Types of Head Mounted Displays
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
The image cannot be displayed. Your computer
Occluded See-thru
Multiplexed
Immersive VR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
virtual object
Virtual World
See-thru AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
see-thru!combiner
Virtual Image superimposed!over real world object
real world
Image source
Optical see-through head-mounted display
Virtual images from monitors
Real World
Optical Combiners
Optical See-Through HMD
Optical see-through HMDs
Sony Glasstron
Virtual Vision VCAP
View Through Optical See-Through HMD
DigiLens
www.digilens.com
Compact HOE Solid state optics Switchable Bragg Grating Stacked SBG Fast switching Ultra compact
Google Glasses
The Virtual Retinal Display
Image scanned onto retina Commercialized through Microvision
Nomad System - www.mvis.com
Strengths of optical AR Simpler (cheaper) Direct view of real world
Full resolution, no time delay (for real world) Safety Lower distortion
No eye displacement but COASTAR video see-through avoids this
Video AR Architecture
Head!Tracker
Host !Processor
Graphics!renderer
Digital!Mixer Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
Head-mounted camera aligned to
display optics
Video!Processor
Video image of real world
Virtual image inset into video of real world
Video see-through HMD Video cameras
Monitors
Graphics
Combiner
Video
Video See-Through HMD
Video see-through HMD
MR Laboratory’s COASTAR HMD (Co-Optical Axis See-Through Augmented Reality) Parallax-free video see-through HMD
TriVisio www.trivisio.com Stereo video input
PAL resolution cameras
2 x SVGA displays 30 degree FOV User adjustable convergence
$6,000 USD
View Through a Video See-Through HMD
Vuzix Display
www.vuzix.com Wrap 920 $350 USD Twin 640 x 480 LCD displays 31 degree diagonal field of view Weighs less than three ounces
Strengths of Video AR True occlusion
Kiyokawa optical display that supports occlusion
Digitized image of real world Flexibility in composition Matchable time delays More registration, calibration strategies
Wide FOV is easier to support
Optical vs. Video AR Summary Both have proponents Video is more popular today?
Likely because lack of available optical products
Depends on application? Manufacturing: optical is cheaper Medical: video for calibration strategies
Eye multiplexed AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Virtual Image inset into!real world scene
real world
Opaque!Image source
Virtual Image ‘inset’ into real
Virtual Vision Personal Eyewear
Virtual image inset into real world
Spatial/Projected AR
Spatial Augmented Reality
Project onto irregular surfaces Geometric Registration Projector blending, High dynamic range
Book: Bimber, Rasker “Spatial Augmented Reality”
Projector-based AR
Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo
Projector
Real objects with retroreflective covering
User (possibly head-tracked)
Example of projector-based AR
Ramesh Raskar, UNC, MERL
Example of projector-based AR
Ramesh Raskar, UNC Chapel Hill
The I/O Bulb
Projector + Camera John Underkoffler, Hiroshi Ishii MIT Media Lab
Head Mounted Projector
Head Mounted Projector Jannick Rolland (UCF)
Retro-reflective Material Potentially portable
Head Mounted Projector
NVIS P-50 HMPD 1280x1024/eye Stereoscopic 50 degree FOV www.nvis.com
HMD vs. HMPD
Head Mounted Display Head Mounted Projected Display
Pico Projectors
Microvision - www.mvis.com 3M, Samsung, Philips, etc
MIT Sixth Sense
Body worn camera and projector http://www.pranavmistry.com/projects/sixthsense/
Other AR Displays
Video Monitor AR
Video cameras Monitor
Graphics Combiner
Video
Stereo glasses
Examples
Virtual Showcase
Mirrors on a projection table Head tracked stereo Up to 4 users Merges graphic and real objects Exhibit/museum applications
Fraunhofer Institute (2001) Bimber, Frohlich
Augmented Paleontology
Bimber et. al. IEEE Computer Sept. 2002
Alternate Displays
LCD Panel Laptop PDA
Handheld Displays Mobile Phones
Camera Display Input
Display Taxonomy
Other Types of AR Display Audio
spatial sound ambient audio
Tactile physical sensation
Haptic virtual touch
Haptic Input
AR Haptic Workbench CSIRO 2003 – Adcock et. al.
Phantom
Sensable Technologies (www.sensable.com) 6 DOF Force Feedback Device
AR Haptic Interface
Phantom, ARToolKit, Magellan
AR Tracking and Registration
Registration Positioning virtual object wrt real world
Tracking Continually locating the users viewpoint
- Position (x,y,z) - Orientation (r,p,y)
Registration
Spatial Registration
The Registration Problem Virtual and Real must stay properly aligned If not:
Breaks the illusion that the two coexist Prevents acceptance of many serious applications
Sources of registration errors Static errors
Optical distortions Mechanical misalignments Tracker errors Incorrect viewing parameters
Dynamic errors System delays (largest source of error)
- 1 ms delay = 1/3 mm registration error
Reducing static errors Distortion compensation Manual adjustments View-based or direct measurements Camera calibration (video)
View Based Calibration (Azuma 94)
Dynamic errors
Total Delay = 50 + 2 + 33 + 17 = 102 ms 1 ms delay = 1/3 mm = 33mm error
Tracking Calculate Viewpoint Simulation
Render Scene
Draw to Display
x,y,z r,p,y
Application Loop
20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms
Reducing dynamic errors (1)
Reduce system lag Faster components/system modules
Reduce apparent lag Image deflection Image warping
Reducing System Lag
Tracking Calculate Viewpoint Simulation
Render Scene
Draw to Display
x,y,z r,p,y
Application Loop
Faster Tracker Faster CPU Faster GPU Faster Display
Reducing Apparent Lag
Tracking Update
x,y,z r,p,y
Virtual Display
Physical Display
(640x480)
1280 x 960
Last known position
Virtual Display
Physical Display
(640x480)
1280 x 960
Latest position
Tracking Calculate Viewpoint Simulation
Render Scene
Draw to Display
x,y,z r,p,y
Application Loop
Reducing dynamic errors (2) Match input streams (video)
Delay video of real world to match system lag
Predictive Tracking Inertial sensors helpful
Azuma / Bishop 1994
Predictive Tracking
Time
Position
Past Future
Can predict up to 80 ms in future (Holloway)
Now
Predictive Tracking (Azuma 94)
Related Documents
