Kinect Based 3D Object Manipulation on a Desktop Display

Kinect Based 3D Object Manipulation on a Desktop Display

Mukund Raj, Sarah H. Creem-Regehr, Kristina M.Rand, Jeanine K. Stefanucci and William B.

Thompson

University of Utah

Introduction

• Controlled experimental evaluation of a Kinect based user interface.

• 3D object manipulation in virtual environment.

• Two variations - with & without self avatars.

2

Motivation

• Availability of low cost gesture recognition hardware.

• 3D graphics on web platform.

3

The LeapMicrosoft Windows Kinect

Nintendo Wii

Avatars

• Self Avatars

• Earlier studies – Inconclusive results on spatial cognition tasks.

4

Questions

• Does self avatar have observable effect on interfaces for object manipulation in virtual environments on a desktop display?

• Are there strong individual differences in the effect?

5

Experimental task

• Orientation matching 3D object by rotating object in virtual world.

6

Interface - Display Conditions

7

• Sphere Condition

• Self Avatar Condition

* Between subjects design used

Interface – Interaction Methods

8

• Swipe – Rotation along hand motion.

• Twist – Rotation about the wrist.

Experimental objects

• Objects

9

Number of objects Trials per object

Training 1 6

Test 12 1

Training Object Test Object #1

Experiment Setup

10

• Kinect

• Orientation Sensor

• Wireless Mouse

Result – Completion Time

• No significant difference between sphere and avatar condition overall

11

Result – Completion Time

• Difference as function of gender and video game experience.

12

Gaming experience / Gender ?

13

Results – Manipulation modes

• Difference in manipulation mode as a function of gender/gaming experience

14

Conclusion

• Effect of self avatar on performance of only a subset of users

• Necessary to check for individual differences in performance data

16

Future Research

• Automatic twist/swipe mode detection• Replace orientation sensor with a low cost

mobile device accelerometer• 6DOF task• Stereographic displays• Symmetrical objects• Tactile Feedback• Comprehension vs. Manipulation

15

This work was supported by the National Science Foundation under Grant No. 1116636

17