GESTURAL COMMUNICATIONS WITH ACCELEROMETER-BASED INPUT DEVICES AND MULTI-MODAL DISPLAYS ul D. Varcholik TIVE Laboratory stitute for Simulation and Training iversity of Central Florida [email protected]James L. Merlo LTC, US Army US Military Academy West Point [email protected]
15
Embed
Paul D. Varcholik ACTIVE Laboratory Institute for Simulation and Training University of Central Florida [email protected] James L. Merlo LTC, US Army.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
GESTURAL COMMUNICATIONS WITH ACCELEROMETER-BASED INPUT DEVICES AND MULTI-MODAL DISPLAYS
Paul D. VarcholikACTIVE LaboratoryInstitute for Simulation and TrainingUniversity of Central [email protected]
Wireless (Bluetooth) 1.2 lbs (w/o battery) Elastic belt 8 tactors at 45-degree
increments
Tactile Patterns Emulating Standard Army Hand Signals (FM 21-60)
TACTONS
Gesture Recognition Machine Learning Algorithms (3 implemented for
evaluation) Linear Classifier AdaBoost Artificial Neural Network (evolved w/ NEAT)
29 Features Based on work by Rubine (1991) on 2D symbol
recognition Example features:
○ Bounding Volume Length○ Min, Max, Median, Mean (X, Y, Z)○ Starting Angle, Total Angle Traversed, Total Gesture Distance
Training & Visualization UI Arbitrary gesture set Left-hand, right-hand,
both-hands 3D animated soldier Text label display Sound display Tactile display Wiimote visualization Data serialization UI Independent of
recognition API
Experiments & Results
Several experiments run to dateDifferent algorithms and gesture setsAccuracy > 94%Classification time < 10ms / gestureLinear classifier best performer (for training
time and classification considered together)AdaBoost (highest accuracy, but slower
training time than linear classifier)ANN w/ NEAT (worst performer – requires
more training data)
Discussion Proved Concept
System capable of accurately converting and transmitting a visual communication mode into a non-visual form.
Wiimote is a convenient and inexpensive device for experimentation. Technology transfers to more robust hardware (e.g. instrumented glove).
Wiimote produces some ambiguous data (e.g. static poses). Additional attachment (e.g. gyroscopes) required for more accuracy.
Experiments indicate promising form of communication – more experiments are needed.
Future Work Determine the maximum number of
gestures that can be accurately recognized Gesture rejection Dynamic mapping between gesture, sound,
and tactile sequence Scenario development for realistic
experimentation (establishing context) Transmitting signal data via RF (currently
sent to local device or via UDP/IP)
USMA Collaboration
CDT Robert Darket, CDT Zachary Schaeffer (Principal Investigators)
Application: TrainingCollect exemplar gestures from SMEsValidate less-experienced soldier’s gestures
against exemplars
Video Demonstration
Questions?
Gestural communications with accelerometer-based input devices and multi-modal displaysPaul D. Varcholik
ACTIVE LaboratoryInstitute for Simulation and TrainingUniversity of Central [email protected]