VR Interfaces – Navigation, Selection and UI Elements By David Johnson.

VR Interfaces – Navigation, Selection and UI Elements

By David Johnson

VR Interfaces

• How do we tell the computer to do things?

• How do we select things?

• How do we navigate around?

VR Interface Challenges

• Intuitive– Make interaction work like the real world– Minimize cognitive overhead

• Augmentation– Give users new capabilities

Quick UI review

• Norman’s Principles of Design– Make things visible– Provide a good conceptual model

• Affordance• Mapping• Constraints• Feedback

Visibility

Good Conceptual Model

• A good conceptual model allows us to predict the effects of our actions

• Without a good model we operate blindly– Simply follow rules without understanding a

reason– No understanding of cause or effect– No recourse when something breaks

• Fridge/freezer controls• Thermostat

Affordances

Mapping

Constraints

• Prevent you from doing what you shouldn’t do– Grey out selections that don’t apply at the

current time

Feedback Examples

• Clicker on your turn signal

• Animated icon while waiting for a web page to load

Why is usability important?

• Poor usability results in– anger and frustration– decreased productivity in the workplace– higher error rates– physical and emotional injury– equipment damage– loss of customer loyalty

2D Interfaces

• Dominant computer interface uses a mouse and graphical elements

Xerox Star (1981)

2D Interfaces

• Why is it a WIMP interface?– Windows– Icons– Menus– Pointer

Xerox Star (1981)

3D Interfaces

• Need to map 2D interfaces to 3D

• Hopefully, create whole new expressive interfaces

3D equivalent of a Mouse?

• Mouse – 2D positioning– Buttons to hold or click

6DOF mouse

• Flying mouse• Fledarmaus• The Bat

• How do you clutch/ratchet?– In 2D, picking up

disables tracking

Menus in Virtual Space

• Cannot easily overlay menus

• “Float” menus in space– Select by raycasting– Keep near user’s head

Jacoby, Ellis 1992

Menus in Virtual Space

• Ring Menus– JDCAD 1993

• Liang

– ISAAC 1995, • Mark Mine

– Rotate hand to moveselection point

Menus in Virtual Space

• Darken, 1994

• Overlaid menus

• Speech selection

Menus in Virtual Space• Pen and Tablet

– Track a tablet and pen– Put 2D menus on

tablet– “Haptic Hand”

Menus in Virtual Space

• Bowman• Pinch Gloves

– Select with thumb to finger

– High-level menu on ND hand

– Secondary menu on D hand

– First tries• Scrolling menu using

pinches or• More items on pinkie

TULIP Menus

• Three-Up, Labels in Palm

• Virtually raise hands• Rotate menus• Put ‘next’ groups on

palms• Users preferred over

floating and tablet menus– Perhaps slower

Menus in Virtual Spaces

• ‘Virtual tricorder’

• Wloka 1995

Gestures

• Symbolic– Cultural meaning (O.K. sign)

• Deictic– Pointing, direct viewer’s attention

• Iconic– Showing an example path with hand

• Pantomimic– Act out the activity

Gestures

• GIVEN (1992)– Neural net recognition– 20 gestures

• Fly, grab, etc.

• Mine– “Physical mnemonics”

• Pull-down menus from near head• Delete by throwing over shoulder

Numerical Input

• Mark Mine

• A digit at a time

• Sliders too imprecise

Text Input

• Bowman

• Pinch glove

• Thumb to home row finger

• Hands in/out to go down/up row

• Rotate to hit extra keys

Basic Navigation Tasks

• Exploration– Untargeted movement– Build internal map

• Positioning– Move to known location

• Maneuvering– Precise positioning of viewpoint– Typically short motions

Natural Interfaces

• Walking

• Biking

• Snowboard

• Swimming

• Issues?

Walking workaround

• Redirected walking– movie

Flying Interfaces

• Flying

• Magic carpet

• Guided navigation– River analogy

• Issues?

Steering Interfaces

• Pointing– Expressive– Hand shake

• Torso

• Gaze-directed– simple

• Physical device

Hand-based Interfaces

• Colin Ware (1990s)– World-in-hand– Eye-in-hand

Point-to-point Travel

• Select a point in a scene– Computer picks path

• Teleport– Bowman et al. found significant spatial

disorientation from teleport

World in Miniature

• User holds dynamic map in one hand

• Navigation is reduced to object positioning

WIM Setup

• Physical props – clipboard and interface ball

Two-handed Flying

• Mark Mine

Fundamental Operations in a UI

• Select an object• Manipulate an object

– Translate– Rotate

• What are some techniques in 2D interfaces?

From the Beginning

• Sutherland and Vickers– Sorcerer’s Apprentice

(1972)

• Track stylus• Selection of vertices

– Intersection of cube at tip of stylus

Pointing: Put That There

• 1979

• Ray from tracked hand

• Speech interface

• movie

Pointing: JD-CAD (1993)

• “Laser gun” from hand– Tracker noise– Harder to select far

away

• Spotlight– Add a cone to the ray– Select objects based

on• Distance from cone

axis• Distance from hand

Silk Cursor

• Replace wireframe selection box with translucent box– Visual cues to

containment

Pointing: Aperture

• Spotlight from eye• Cone angle based on

distance from hand to eye

• Selection modified by hand orientation

Pointing: Flexible Pointer

• Two-handed• Hand direction bends

pointer• Can select occluded

objects• movie

Hand: GoGo Interaction (1996)• Go-Go uses Non-Linear mapping between virtual and real hand• Control-display ratio• Stretch go-go variation• Pros:

– Extended reach when needed– Direct manipulation

• Cons:– Reach still limited by arm length– Precision suffers when reach is extended (low level of control)

Movie

Image Plane Techniques

• Point or gesture at an objects projection onto the viewing plane– “head-crusher”

• Kids in the hall– “Sticky finger”

• Similar to ray casting• Pros:

– Very intuitive– Allows user to reach objects at

an arbitrary distance• Cons:

– Limited by the need for line of sight

– Can be fatiguing– Virtual hand may obscure small

objects

Two Handed and Body-Centered Interaction

• What can you do with two hands?• What if you use your body as a

reference point?• Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin

(1997). Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Angeles, CA.

• “Scaled-space grab”

HOMER technique

Hand-Centered Object Manipulation Extending Ray-Casting

• Select: ray-casting• Manipulate: hand• Translation

proportional to initial object distance

Time

World in Miniature

• User holds dynamic map in one hand

• Objects can be moved in map

• What about fine positioning?

• What about selection of small objects?

Voodoo Dolls

• User creates map with image plane selection

Summary

• Ergonomics an issue

• Usability still low

• Standard GUI elements translate poorly