3D Interaction Techniques for Virtual Environments
Post on 30-Dec-2015
34 Views
Preview:
DESCRIPTION
Transcript
(C) 2005 Doug Bowman, Virginia Tech 2
Technique Classificationby metaphor
VE manipulation techniquesExocentric metaphor
Egocentric metaphor
World-In-Miniature
Virtual Hand metaphor
Virtual Pointer metaphor
Go-GoIndirect, stretch Go-Go
"Classical" virtual hand
Ray-castingApertureFlashlightImage plane
Scaled-world grab
(C) 2005 Doug Bowman, Virginia Tech 3
Manipulation metaphors I
Ray casting little effort requiredExact positioning and orienting very difficult
(lever arm effect)
(C) 2005 Doug Bowman, Virginia Tech 4
5.4.2 Interacting by Pointing Selection process
When the vector defined by the direction of pointing intersects a virtual object, the user can select it by triggering event that confirms the selection.
Examples of triggers are buttons and voice command A number of pointing techniques have been reported
How the pointing direction is defined By the shape of the selection volume
Pointing is a powerful selection technique Pointing, however, is generally a very poor positioning
(Manipulation) technique Rotation can be effectively accomplished only about one
axis. Expressive 6-DOF manipulation is therefore impossible
(C) 2005 Doug Bowman, Virginia Tech 5
5.4.2 Interacting by Pointing
Simple ray-casting technique
Two-handed pointing
Flashlight / Aperture techniques
Image-plane techniques
Fishing-reel techniques
(C) 2005 Doug Bowman, Virginia Tech 6
Ray Casting (I) The user points at objects with a virtual ray
that defines the direction of pointing. Attached to the virtual hand controlled by a 6-DOF
sensor in immersive environment. Attached to a 3D widget controlled by a mouse in a
desktop 3D environment. Pointing direction
p() = h + P, where P = the direction of user’s virtual hand, h= 3D position of the virtual hand
More than one object can be intersected by the line p(), and only one closest to the user should be selected.
(C) 2005 Doug Bowman, Virginia Tech 7
Ray Casting (II) The shape of the ray
Short line segment; Figure 5.4 Infinitely long virtual ray; better visual feedback
Virtual ray casting is a very powerful selection technique Except when a very high precision of selection is
required; selecting small or faraway objects At close range, ray-casting is perhaps the most
simple and efficient selection technique.
(C) 2005 Doug Bowman, Virginia Tech 8
Two-Handed Pointing Two handed technique: one hand specifies the origin
of the virtual ray, while the other hand specifies where the ray is pointing to
p() = hl + ( hr – hl ) , where hr and hl = 3D position of the right and left virtual hands respectively.
Disadvantage Both hands must be tracked.
Advantage Allow for richer and more effective pointing interaction Curve the virtual pointer by twisting the hand slightly. Fig. 5.5 참조 .
(C) 2005 Doug Bowman, Virginia Tech 9
Flashlight and Aperture Techniques (I) Spotlight or flashlight technique
It replaces the virtual ray with a conic selection volume, with the apex of the cone at the input device; Fig. 5.6
Objects that fall within this selection cone can be selected. Easy selection of small objects even when they are
located far from the user Disambiguation
When more than one object falls into the spot light. Disadvantages:
When selection of small objects or tightly grouped objects
(C) 2005 Doug Bowman, Virginia Tech 10
Flashlight and Aperture Techniques (II) Aperture Technique
Modification of the flashlight technique and improve it. Selection line P() p() = e + (h – e ) , e = virtual viewpoint
h = hand The user can interactively control the spread angle of the
selection volume simply by bringing the hand sensor closer or moving it father away; Figure 5.6
Simplify selection of virtual objects by using the orientation of the pointer around a central avis as an additional disambiguation metric. Figure 5.7b
Used easily in both desktop and immersive VEs.
(C) 2005 Doug Bowman, Virginia Tech 11
Image-Plane Techniques The user selects and manipulates 3D objects by
touching and manipulating their 2D projections on a virtual image plane located in front of the user. Figure 5.8
Sticky-finger technique Figure 5.8
Head-crusher technique Another Image-plane tech. With a data glove devices With two fingers, his thumb and index fingers
Allows the user to modify the orientation of 3D objects, but their distance from the user can not be directly controlled. Mine’s scaled-world grab, Pierce’s Voodoo Doll technique
(C) 2005 Doug Bowman, Virginia Tech 12
Fishing-Reel Technique
The ray direction is controlled by the spatial movements of user’s hand,
While distance is controlled by other means. Controlling the length of the virtual ray.A simple mechanical slider or a pair of
buttons added to the tracking device
(C) 2005 Doug Bowman, Virginia Tech 13
Manipulation metaphors II
Simple virtual handNatural but limited
(C) 2005 Doug Bowman, Virginia Tech 14
5.4.3 Direct Manipulation: Virtual Hand Techniques
3D cursor 3D model of human hand Semitransparent volumetric cursor
Selection The user intersects 3D cursor with the target of selection
Trigger technique : to pick it up Button press, voice command, hand gesture The object is attached to the virtual hand and can be easily
translated and rotated within V.E Release
The user release it with another trigger.
(C) 2005 Doug Bowman, Virginia Tech 15
Virtual Hand Interaction Techniques
Simple (“Classical” ) virtual hand technique
Go-Go technique
Indirect Go-Go
(C) 2005 Doug Bowman, Virginia Tech 16
Simple Virtual Hand (I) Direct mapping :
transfer functions or control-display gain functionpv = αpr , Rv = Rr (Eq. 5.4)
pr Rr are the position and orientation(3*3 matrix) of the user’s real hand.
pv , Rv are the corresponding position and orientation of the virtual handα is a scaling ratio to match the scales of the real
and virtual coordinate systems.
rv pp
(C) 2005 Doug Bowman, Virginia Tech 17
Simple Virtual Hand (II)
Scaling rotationIt is useful to “scale” 3D device rotations similar to
the way we scale translations
Fundamental problemIn order to select objects located further away, the
user must employ a travel technique. Inconvenient and increase the complexity of the 3D UI.
rv pp
(C) 2005 Doug Bowman, Virginia Tech 18
Go-Go Interaction TechniqueWhile the user’s real hand is close to the user (the distance to the hand is smaller than threshold D), the mapping is one to one, the movementof the virtual hand correspondto the real hand movements
As the user extends her handbeyond D, the mapping becomes nonlinear and the virtual arm “grows”, thus permitting the user to access and manipulate remote objects. C1 continuity is ensured.
(C) 2005 Doug Bowman, Virginia Tech 19
Go-Go IT
Different mapping function can be used to achieve a different control-display gain between the real and virtual hands.
Advantages: Provide direct, seamless, 6-DOF object manipulation both
close to the user and at a distance. Disadvantages:
As the distance increases, the technique maps small movements of the user’s hand into large movements of the virtual hand, which complicates precise positioning at a distance.
(C) 2005 Doug Bowman, Virginia Tech 20
5.4.5 Combining Techniques
Aggregation of techniques Mechanism for choosing the desired manipulation
technique from a limited set of possible options
Technique integration Combining techniques in which the interface switches
transparently between interaction techniques depending on the current task context.
Selection 과 manipulation 이 반복 적용되므로 , 각 mode 에서 최선의 방법들을 선택하여 적용한다 .
(C) 2005 Doug Bowman, Virginia Tech 21
HOMER technique
Hand-Centered Object Manipulation Extending Ray-Casting
Select: ray-casting Manipulate: hand
Time
(C) 2005 Doug Bowman, Virginia Tech 22
Manipulation metaphors III
HOMER (ray-casting + arm-extension)Easy selection & manipulationExpressive over range of distancesHard to move objects away from you
(C) 2005 Doug Bowman, Virginia Tech 23
HOMER implementation Requires torso position t Upon selection, detach virtual hand from tracker,
move v. hand to object position in world CS, and attach object to v. hand (w/out moving object)
Get physical hand position h and distance
dh = dist(h, t)
Get object position o and distance
do = dist(o, t)
(C) 2005 Doug Bowman, Virginia Tech 24
HOMER implementation (cont.)
Each frame:Copy hand tracker matrix to v. hand matrix (to
set orientation)Get physical hand position hcurr and distance:
dh-curr = dist(hcurr, t)V. hand distance
Normalize torso-hand vectorV. hand position vh = t + dvh*(thcurr)
dvh dh curr dodh
thcurr hcurr thcurr t
(C) 2005 Doug Bowman, Virginia Tech 25
Manipulation metaphors IV
World-in-miniature (WIM)All manipulation in reachDoesn’t scale well, indirect
Scaled-world grabEasy, natural manipulationUser discomfort with use
(C) 2005 Doug Bowman, Virginia Tech 26
5.4.4 World-in-miniature (WIM)
“Dollhouse” world held in user’s hand
Miniature objects can be manipulated directly
Moving miniature objects affects full-scale objects
(C) 2005 Doug Bowman, Virginia Tech 27
World-in-Miniature (WIM) Provides the user with miniature handheld model of VE, which is
an exact copy of the VE at a small scale. Careful use of back face culling techniques. Only the “inside” of
the walls of the room model should be rendered. WIM combine navigation with manipulation. Although WIM works well for small and medium-sized
environments, using WIM in a very large environment would require an extreme scale factor, resulting in very small object copies in the WIM. This would make accurate selection and manipulation extremely difficult.
It has been successfully used in 3D interfaces for Augmented reality, in desktop 3D Uis. In fact, it can be considered a 3D generalization of the traditional overview maps that are often used in 3D games.
(C) 2005 Doug Bowman, Virginia Tech 28
WIM implementation
Root
head hand room
table
Root
head hand room
tableWIM room(scaled)
table copy
(C) 2005 Doug Bowman, Virginia Tech 29
Scale-world Grab Based on HOMER Selection : Image-plane selection tech. is used. Manipulation :
Scale down the entire VE around the user’s virtual viewpoint switching into manipulation mode
In HOMER, scaling the user’s hand motion The scaling coefficient s = Dv / Do where Dv is the distance
from the virtual viewpoint to the virtual hand, and Do is the distance from the virtual viewpoint to the selected object.
Well for operations at a distance but not effective when the user wants to pick up an object located within arm’s reach and move it farther away.
(C) 2005 Doug Bowman, Virginia Tech 30
Voodoo Dolls To overcome the scaling approach such as HOMER, scaled-
World grab when the user needs to move local objects farther away.
Image-plane and WIM with a pair of pinch gloves Manipulate virtual objects indirectly using miniature handheld
copies of objects called “dolls”; Figure 5.13 Selecting the target object with an image-plane technique, which
creates the dolls representing the target objects and places them in the user’s hand.
The doll in her non-dominant hand : the corresponding virtual object does not move when the user moves this doll.
To start manipulation, the user simply passes the doll into dominant hand.
(C) 2005 Doug Bowman, Virginia Tech 31
Voodoo Dolls Advantages:
Powerful IT allowing to perform some sophisticated tasks, such as the manipulation of moving, animated objects
Key idea is very universal and important insight Separating functionality depending on the dominant and
non-dominant hands
Disadvantages Increases H/W demand Direct application to a desktop might be difficult.
(C) 2005 Doug Bowman, Virginia Tech 32
Technique Classification by metaphor
VE manipulation techniquesExocentric metaphor
Egocentric metaphor
World-In-Miniature
Virtual Hand metaphor
Virtual Pointer metaphor
Go-GoIndirect, stretch Go-Go
"Classical" virtual hand
Ray-castingApertureFlashlightImage plane
Scaled-world grab
(C) 2005 Doug Bowman, Virginia Tech 33
Technique Classificationby components
Manipulation
Object Attachment
Object Position
Object Orientation
Feedback
attach to handattach to gazehand moves to objectobject moves to handuser/object scaling
no control1-to-N hand to object motionmaintain body-hand relationother hand mappingsindirect control
no control1-to-N hand to object rotationother hand mappingsindirect control
graphicalforce/tactileaudio
(C) 2005 Doug Bowman, Virginia Tech 34
Evaluation: positioning task
Ray casting effective if the object is repositioned at constant distance
Scaling techniques (HOMER, scaled world grab) difficult in outward positioning of objects: e.g. pick an object located within reach and move it far away
If outward positioning is not needed then scaling techniques might be effective
(C) 2005 Doug Bowman, Virginia Tech 35
Evaluation: orientation task
Setting precise orientation can be very difficult
Shape of objects is important
Orienting at-a-distance harder than positioning at-a-distance
Techniques should be hand-centered
(C) 2005 Doug Bowman, Virginia Tech 36
Manipulation notes
No universally best technique
Constraints and reduced DOFs
Naturalism not always desirable
If VE is not based in the real, design it so manipulation is optimized
(C) 2005 Doug Bowman, Virginia Tech 37
Manipulation enhancements
Constraints
2-handed manipulation
Haptic feedback
Multi-modal manipulation
(C) 2005 Doug Bowman, Virginia Tech 38
Implementation issues for manipulation techniques
Integration with selection technique
Disable selection and selection feedback while manipulating
What happens upon release?
(C) 2005 Doug Bowman, Virginia Tech 40
Common manipulation techniques
Simple virtual hand
HOMER
Scaled-world grab
World-in-miniature
(C) 2005 Doug Bowman, Virginia Tech 41
Simple virtual hand techniqueAttach object to virtual hand,
by making object a child of the hand (w/out moving object)
On release, reattach object to world (w/out moving object)
Also applies to Go-Go (and other arm-extension techniques) and ray-casting
Root
head hand building
Root
head hand
building
(C) 2005 Doug Bowman, Virginia Tech 42
HOMER techniqueHand-CenteredObject ManipulationExtending Ray-Casting
Select: ray-casting
Manipulate: hand
Time
1.0 m
0.3 m
2.0 m
0.6 m
torso physicalhand
torso physicalhand
(C) 2005 Doug Bowman, Virginia Tech 43
HOMER implementationRequires torso position t
Upon selection, detach virtual hand from tracker, move v. hand to object position in world CS, and attach object to v. hand (w/out moving object)
Get physical hand position h and distance
dh = dist(h, t)
Get object position o and distance do = dist(o, t)
(C) 2005 Doug Bowman, Virginia Tech 44
HOMER implementation (cont.)Each frame:
Copy hand tracker matrix to v. hand matrix (to set orientation)
Get physical hand position hcurr and distance:
dh-curr = dist(hcurr, t)
V. hand distance
Normalize torso-hand vector V. hand position vh = t + dvh*(thcurr)
th
thth
curr
currcurr
h
ocurrhvh d
ddd
(C) 2005 Doug Bowman, Virginia Tech 45
Scaled-world grab techniqueOften used w/ occlusion
At selection, scale user up (or world down) so that v. hand is actually touching selected object
User doesn’t notice a change in the image until he moves
(C) 2005 Doug Bowman, Virginia Tech 46
Scaled-world grab implementationAt selection:
Get world CS distance from eye to hand deh
Get world CS distance from eye to object deo
Scale user (entire user subtree) uniformly by deo / deh
Ensure that eye remains in same position Attach selected object to v. hand (w/out moving object)
At release: Re-attach object to world (w/out moving object) Scale user uniformly by deh / deo
Ensure that eye remains in same position
(C) 2005 Doug Bowman, Virginia Tech 47
World-in-miniature (WIM) technique
“Dollhouse” world held in user’s hand
Miniature objects can be manipulated directly
Moving miniature objects affects full-scale objects
Can also be used for navigation
(C) 2005 Doug Bowman, Virginia Tech 48
WIM implementation
Root
head hand room
table
Root
head hand room
tableWIM room(scaled)
table copy
(C) 2005 Doug Bowman, Virginia Tech 49
WIM implementation (cont.)
On selection: Determine which full-scale object
corresponds to the selected miniature object Attach miniature object to v. hand (w/out
moving object)Each frame:
Copy local position matrix of miniature object to corresponding full-scale object
top related