COSC 426: Augmented Reality Mark Billinghurst [email protected] July 23 rd 2014 Lecture 2: AR Technology [email protected]
2014 COSC 426 Lecture 2: Augmented Reality Technology
Sep 08, 2014
This is the second lecture in the COSC 426
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
COSC 426: Augmented Reality
Mark Billinghurst
July 23rd 2014
Lecture 2: AR Technology
Recap
Augmented Reality Definition Defining Characteristics [Azuma 97]
Combines Real and Virtual Images - Both can be seen at the same time
Interactive in real-time - Virtual content can be interacted with
Registered in 3D - Virtual objects appear fixed in space
What is not Augmented Reality?
Location-based services Barcode detection (QR-codes) Augmenting still images Special effects in movies … … but they can be combined with AR!
AR vs VR Virtual Reality: Replaces Reality
Scene Generation: requires realistic images Display Device: fully immersive, wide FOV Tracking and Sensing: low accuracy is okay
Augmented Reality: Enhances Reality Scene Generation: minimal rendering okay Display Device: non-immersive, small FOV Tracking and Sensing: high accuracy needed
Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real Environment
Augmented Reality (AR)
Augmented Virtuality (AV)
Virtual Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
A Brief History of AR (1) 1960’s: Sutherland / Sproull’s
first HMD system was see-through
History Summary 1960’s – 80’s: Early Experimentation 1980’s – 90’s: Basic Research
Tracking, displays
1995 – 2005: Tools/Applications Interaction, usability, theory
2005 - : Commercial Applications Games, Medical, Industry
Google Searches for AR
2008 - Browser Based AR Flash + camera + 3D graphics High impact
High marketing value
Large potential install base 1.6 Billion web users
Ease of development Lots of developers, mature tools
Low cost of entry Browser, web camera
2005 - Mobile Phone AR Mobile Phones
camera processor display
AR on Mobile Phones Simple graphics Optimized computer vision Collaborative Interaction
2009 - Outdoor Information Overlay Mobile phone based Tag real world locations
GPS + Compass input Overlay graphics data on live video
Applications Travel guide, Advertising, etc
Wikitude, Layar, Junaio, etc.. Android based, Public API released
AR Today Key Technologies Available
- Robust tracking (Computer Vision, GPS/sensors) - Display (Handheld, HMDs) - Input Devices (Kinect, etc) - Developer tools (Qualcomm, Metaio, ARTW)
Commercial Business Growing - Gaming, GPS/Mobile, Online Advertisement
• >$5 Billion USD by 2016 (Markets andMarkets) • >$1.5 Billion USD in Mobile AR by 2014 (Juniper Research)
Sample AR Applications
Applications
Medicine Manufacturing Information overlay Architecture Museum Marketing Gaming
Applications: medical “X-ray vision” for surgeons Aid visualization, minimally-invasive operations.
Training. MRI, CT data. Ultrasound project, UNC Chapel Hill.
Courtesy UNC Chapel Hill
Medical AR Trials Sauer et al. 2000 at Siemens
Corporate Research, NJ Stereo video see through
F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial, MICCAI 2002
Assembly and maintenance
© 1993 S. Feiner, B. MacIntyre, & D. Seligmann, Columbia University
© 1996 S. Feiner, B. MacIntyre, & A. Webster, Columbia University
PS3 - Eye of Judgment (2007) Computer Vision Tracking Card based battle game Collaborative AR October 24th 2007
AR Books – Markerless Tracking
AR Annotations
Columbia University
HRL
© 1993 S. Feiner, B. MacIntyre, M. Haupt, & E. Solomon, Columbia University
© 1997 S. Feiner, B. MacIntyre, T. Höllerer, & A. Webster, Columbia University
Broadcast TV
Interactive Museum Experiences BlackMagic
Virtual America’s Cup 410,000 people in six months
MagicPlanet TeManawa science museum Virtual Astronomy Collaborative AR experience
AR Volcano Interactive AR kiosk Scienceworks museum, Melbourne
Digital Binocular Station
http://www.DigitalBinocularStation.com/
Museum Archeology LifePlus (2002-2004)
Natural feature tracking Virtual characters Mobile AR system
Archeoguide (2000-2002) Cultural heritage on-site guide Hybrid tracking Virtual overlay
Sales and Marketing Connect with brands and branded objects Location Based Experiences
Lynx Angels
Web based Rayban glasses
Mobile Ford Ka campaign
Print based Red Bull Magazine
Summary AR technology can be used to develop a wide
range of applications Promising application areas include
Games Education Engineering Medicine Museums Etc..
AR Experience Design
“The product is no longer the basis of value. The
experience is.”
Venkat Ramaswamy The Future of Competition.
Experience Economy
experiences
services
products
components
Valu
e
Sony CSL © 2004
Gilmore + Pine: Experience Economy
Function
Emotion
Good Experience Design Reactrix
Top down projection Camera based input Reactive Graphics No instructions No training
Improve the experience of picking up rubbish?
World’s Deepest Rubbish Bin
The Fun Theory – http://www.funtheory.com
Improve the experience of walking up stairs?
Musical Stairs
The Fun Theory – http://www.funtheory.com
Apple: The Value of Good Design
Good Experience Design Dominates Markets
iPod Sales 2002-2007
Using the N-gage
SideTalking http://www.sidetalkin.com
Interaction Design
“Designing interactive products to support people in their everyday and working lives” Preece, J., (2002). Interaction Design
Design of User Experience with Technology
Higher in the value chain than product design
Interaction Design involves answering three questions: What do you do? - How do you affect the world? What do you feel? – What do you sense of the world? What do you know? – What do you learn?
Interaction Design is All About You
Users should be involved throughout the Design Process
Consider all the needs of the user
Interaction Design Process
experiences
applications
tools
components
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability
Summary In order to build AR applications you need to
focus on the user experience Great user experience is based on
Low level AR component technology Authoring tools Application/Interaction design User experience texting
AR Technology
experiences
applications
tools
components
Sony CSL © 2004
Building Compelling AR Experiences
Display, Tracking
Core Technologies Combining Real and Virtual Images
• Display technologies Interactive in Real-Time
• Input and interactive technologies Registered in 3D
• Viewpoint tracking technologies Display
Processing
Input Tracking
AR Displays
AR Displays
e.g. window reflections
Virtual Images seen off windows
e.g. Reach-In
Projection CRT Display using beamsplitter
Not Head-Mounted
e.g. Shared Space Magic Book
Liquid Crystal Displays LCDs
Head-Mounted Display (HMD)
Primarily Indoor Environments
e.g. WLVA and IVRD
Cathode Ray Tube (CRT) or Virtual Retinal Display (VRD)
Many Military Applications & Assistive Technologies
Head-Mounted Display (HMD)
e.g. Head-Up Display (HUD)
Projection Display Navigational Aids in Cars
Military Airborne Applications
Not Head Mounted (e.g. vehicle mounted)
Primarily Outdoor (Daylight) Environments
AR Visual Displays
Display Technologies
Types (Bimber/Raskar 2003) Head attached
• Head mounted display/projector Body attached
• Handheld display/projector Spatial
• Spatially aligned projector/monitor
Display Taxonomy
Head Mounted Displays
Head Mounted Displays (HMD) - Display and Optics mounted on Head - May or may not fully occlude real world - Provide full-color images - Considerations
• Cumbersome to wear • Brightness • Low power consumption • Resolution limited • Cost is high?
Key Properties of HMD Field of View
Human eye 95 degrees horizontal, 60/70 degrees vertical
Resolution > 320x240 pixel
Refresh Rate Focus
Fixed/manual
Power Size
Types of Head Mounted Displays
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
The image cannot be displayed. Your computer
Occluded See-thru
Multiplexed
Immersive VR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
virtual object
Virtual World
See-thru AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
see-thru!combiner
Virtual Image superimposed!over real world object
real world
Image source
Optical see-through head-mounted display
Virtual images from monitors
Real World
Optical Combiners
Optical See-Through HMD
Epson Moverio BT-200
▪ Stereo see-through display ($700) ▪ 960 x 540 pixels, 23 degree FOV, 60Hz, 88g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer
View Through Optical See-Through HMD
The Virtual Retinal Display
Image scanned onto retina Commercialized through Microvision
Nomad System - www.mvis.com
Strengths of optical see-through AR Simpler (cheaper) Direct view of real world
Full resolution, no time delay (for real world) Safety Lower distortion
No eye displacement but COASTAR video see-through avoids this
Video AR Architecture
Head!Tracker
Host !Processor
Graphics!renderer
Digital!Mixer Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
Head-mounted camera aligned to
display optics
Video!Processor
Video image of real world
Virtual image inset into video of real world
Video see-through HMD Video cameras
Monitors
Graphics
Combiner
Video
Video See-Through HMD
Vuzix Wrap 1200DXAR
▪ Stereo video see-through display ($1500) ■ Twin 852 x 480 LCD displays, 35 deg. FOV ■ Stereo VGA cameras ■ 3 DOF head tracking
View Through a Video See-Through HMD
Strengths of Video See-Through AR True occlusion
Virtual images can block view of real world
Digitized image of real world Flexibility in composition Matchable time delays More registration, calibration strategies
Wide FOV is easier to support
Optical vs. Video AR Summary Both have proponents Video is more popular today?
Likely because lack of available optical products
Depends on application? Manufacturing: optical is cheaper Medical: video for calibration strategies
Eye multiplexed AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Virtual Image inset into!real world scene
real world
Opaque!Image source
Virtual Image ‘inset’ into real
Google Glass
View Through Google Glass
Vuzix M-100
▪ Monocular multiplexed display ($1000) ■ 852 x 480 LCD display, 15 deg. FOV ■ 5 MP camera, HD video ■ GPS, gyro, accelerometer
Display Types
▪ Curved Mirror ▪ off-axis projection ▪ curved mirrors in front of eye ▪ high distortion, small eye-box
▪ Waveguide ▪ use internal reflection ▪ unobstructed view of world ▪ large eye-box
See-through thin displays
▪ Waveguide techniques for thin see-through displays ▪ Wider FOV, enable AR applications ▪ Social acceptability
Opinvent Ora
Waveguide Methods
See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable
Holographic Hologram diffracts light Limited FOV Colour bleeding
Diffractive Slanted gratings Total internal reflection Costly, small FOV
Waveguide Methods
See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable
Clear-Vu Reflective Several reflective elements Thinner light guide Large FOV, eye-box
Reflective Simple reflective elements Lower cost Size is function of FOV
Comparison Chart
Related Documents
