Building Usable AR Interfaces Mark Billinghurst HIT Lab NZ University of Canterbury
May 19, 2015
Building Usable AR Interfaces
Mark Billinghurst HIT Lab NZ
University of Canterbury
The Vision of AR
To Make the Vision Real.. Hardware/software requirements
Contact lens displays Free space hand/body tracking Speech/gesture recognition Etc..
Most importantly Usability
Usability Issues Perception
How to make images appear part of real world?
Interaction How to pick content in mid air?
Social Do I look stupid?
Cultural Wave up or down?
Cognitive How do I remember where everything is?
“The product is no longer the basis of value. The
experience is.”
Venkat Ramaswamy The Future of Competition.
experiences
services
products/tools
components
Valu
e Gilmore + Pine: Experience Economy
Function
Emotion
Build Experiences NOT Applications!
experiences
applications
tools
components
Sony CSL © 2004
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability ??
The Interaction Design Process
Understand Your Users Workshops or focus groups
Group interviews/activities
Observations Spending time with users in day to day tasks
Questionnaires Looking for specific information
Interviews Good for exploring issues, using props
Documentation Procedures and rules written down in manuals
Consider the Whole User
AR as Perception Problem Goal of AR to fool human senses – create
illusion that real and virtual are merged Depth
Size Occlusion Shadows Relative motion Etc..
TAT Augmented ID
AR Rapid Development
Prototyping and User Testing Low Fidelity Prototyping Sketches Paper Prototyping Post-It Prototyping PowerPoint Prototyping
High Fidelity Prototyping Wikitude, Junaio, Layar, BuildAR etc
POST IT PROTOTYPING
First Dra)
Camera View with 3D
Second Dra) Third Dra)
• Selec8on highlighted in blue
• Home bu=on added for easy naviga8on to main menu
POWERPOINT PROTOTYPING Benefits • Used for User Tes8ng • Interac8ve • Func8onali8es work when following the story of Scenario 1 • Quick • Easy arrangement of slides
User Tes8ng • Par8cipants found • 15 minute sessions screen captured
• ‘Talk Allowed’ technique used • Notes taken • Post-‐Interview
BuildAR
http://www.buildar.org/ Stand alone application Visual interface for AR model viewing application Enables non-programmers to build AR scenes
Interface Components Physical components Display elements
- Visual/audio
Interaction metaphor
Physical Elements
Display Elements
Interaction Metaphor Input Output
AR Design Space
Reality Virtual Reality
Augmented Reality
Physical Design Virtual Design
AR Lens Design Principles Physical Components
Lens handle - Virtual lens attached to real object
Display Elements Lens view
- Reveal layers in dataset
Interaction Metaphor Physically holding lens
AR Chemistry (Fjeld 2002)
Tangible AR chemistry education
Case Study: LevelHead
Block based game
Case Study: LevelHead
Physical Components Real blocks
Display Elements Virtual person and rooms
Interaction Metaphor Blocks are rooms
Goal: An AR application to test molecular structure in chemistry
Physical Components Real book, rotation cube, scoop, tracking markers
Display Elements AR atoms and molecules
Interaction Metaphor Build your own molecule
AR Chemistry Input Devices
Natural Hand Interaction
Using bare hands to interact with AR content MS Kinect depth sensing Real time hand tracking Physics based simulation model
Evaluation Need for more evaluation
2008 -10% AR papers in IEEE, ACM had any evaluation
Informal Pilot, ‘quick and dirty’
Formal Lab studies, field studies, heuristic
2D vs. AR Navigation?
VS
HIT Lab NZ Test Platform – AR View
HIT Lab NZ Platform – Map View
Distance and Time
No significant differences
Paths Travelled
Red – AR Blue – AR + Map Yellow - Map
Navigation Behaviour Depends on interface
Map doesn’t show short cuts
Survey Responses
User Comments AR
“you don't know exactly where you are all of the time.” “using AR I found it difficult to see where I was going”
Map “you were able to get a sense of where you were” “you are actually able to see the physical objects around you”
AR+MAP “I used the map at the beginning to understand where the
buildings were and the AR between each point” “You can choose a direction with AR and find the shortest way
using the map.”
Building Usable AR Interfaces Understand user needs
Consider whole user needs - Physical, emotional, cognitive, social, cultural
Perceptual issues
Design for those needs Rapid prototyping Virtual, physical elements, interaction metaphor
Test your design Formal, informal testing
More Information
• Mark Billinghurst – [email protected]
• Website – www.hitlabnz.org – www.buildar.org