Template for Presentation _CSE

Department of Computer Science & Engineering, DSCE

WELCOME


MEMS ACCELEROMETER BASED NON-SPECIFIC USER HAND GESTURE RECOGNITION

Vishal Bhaskar1DS10CS128


CONTENTS• INTRODUCTION

• GESTURE MOTION ANALYSIS

• SENSING SYSTEM

• SYSTEM WORK FLOW

• GESTURE SEGMENTATION

• MODEL 1 : BASED ON SIGN SEQUENCE AND HOPFIELD NETWORK.

• MODEL 2 : BASED ON VELOCITY INCREMENT

• MODEL 3 : BASED ON SIGN SEQUENCE AND TEMPLATE MATCHING.

• EXPERIMENTAL RESULTS

• ADVANTAGES

• APPLICATIONS

• CONCLUSION


INTRODUCTION• Human – Machine Interactions

• Gesture Recognition

• Physical Gestures 7 Hand Gestures MEMS Accelerometer3 Models based on time domain

2 Methods o Vision – Based ( Limitation ) o Accelerometer Based


EXISTING SYSTEM• iRobot Ava 500(Telerobotics)

• Gesture controlled TVs

• Nintendo wii,Xbox Kinect (Gaming consoles)


• 3 Gesture Recognition Models

• 7 Gestures

• Inputted to MEMS 3 – Accelerometer

• Gesture Segmentation Algorithm

• 100’s of data to 8 number code

• Gesture Recognition

Sign Sequence & Hopfield Based Velocity Increment Based Sign Sequence & Template Matching Based

Up, down, left, right, tick, circle, cross

PROPOSED SYSTEM


BLOCK DIAGRAM

SENSING CHIP

DATA PROCESSING

GESTURE SEGMENTATION

GESTURE RECOGNITION

GESTURE INPUT

RECOGNISED GESTURE OUTPUT


GESTURE MOTION ANALYSIS

Fig 2 : Gesture up motion decomposition

• Motion in vertical plane ( x – z plane)

• Accelerations on x-z plane

• Up Gesture

Up Gesture

Circle Gesture

o X axis : no accelerationo Z axis : negative – positive – negative

o X axis : positive – negative – positive o Z axis : negative – positive – negative - positive

Velocity zero at pt. 1 & 2 Sign changes at pt. 3 & 4

Fig 1 : Coordinate System


Fig 3 : Predicted velocity and acceleration in the z-axis

Fig 4 : Real acceleration plot

• One Axis – up & down, left & right

• Two Axis – tick, circle, cross (complex)

• Acceleration changes in z axis

• Real acceleration is the same with the prediction

• Unique acceleration pattern

1 to 3 :-ve; V changes from 0 to max. at 3 3 to 4 :+ve; V changes from -ve to +ve & max. at pt 4 4 to 1 :-ve; V changes from +ve to zero


SENSING SYSTEM

Fig 5 : Sensing System

• MEMS 3 – axes acceleration sensing chip

• Data management chip

• Bluetooth Wireless data chip


• MEMS ACCELEROMETER??? Micro Electro-Mechanical Systems Combination of mechanical functions & electronic functions on same chip Measures acceleration forces


SYSTEM WORK FLOW


Fig 6 : Motions of seven gestures

GESTURE SEGMENTATION

• DATA ACQUISITION

Horizontally place sensing device

Time interval not less than 0.2sec

Perform Gestures as shown in Fig 6


• GESTURE SEGMENTATION

DATA PREPROCESSING

2 Processes

o Remove vertical axis offsets by subtracting data points from mean value

o Filter to eliminate noise


SEGMENTATIONo Find terminal points

o We need

o 2 x n matrices generated

o Compare max. acceleration b/w terminal points with its mean valueo No. of columns = No. of gestures

Amplitude of points Point separation Mean value Distance from nearest intersection Sign variation b/w 2 successive points


MODEL ONE : GESTURE RECOGNITION BASED ON SIGN

SEQUENCE AND HOPFIELD NETWORK


Fig 8 : Sign sequence generation

GESTURE RECOGNITION• FEATURE EXTRACTION

Examine the sign of the first mean point

of a gesture

Store in gesture code

Detect no. of sign changes

Store the alternate signs in sequence in

the gesture code

Code for the gesture in fig 8 is 1, -1, 1, -1


• GESTURE ENCODING

Max. no. of signs for 1 gesture on 1 axis is 4

Eight numbers in one gesture code

Hopfield network can take only 1 & -1 as inputs

+ve, -ve sign and zero are encoded as “1 1”, “-1 -1” and “1 -1”

Each gesture has a unique 16 - number code


• HOPFIELD NETWORK AS ASSOCIATIVE MEMORY

Recovery mechanism

Weight matrix is constructed

sp - Pattern to be stored P - Number of patterns I - Identity matrix

npTPp

p

P sPIssw 1,1,)(1

qsv )0(qqTpp

p

p Psssswvu

)()0()1(

1

1

))(sgn()( nunv outputnv )(


• GESTURE COMPARISON

Gesture code is compared with the standard gesture codes Difference b/w the two codes is calculated Smallest difference indicates the most likely gesture

Fig 7 : Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-

circle-cross.


Table 1: Standard patterns for the seven gestures


MODEL TWO: GESTURE RECOGNITION BASED ON VELOCITY INCREMENT

• Model deals with complex gestures

• Area bounded by acceleration curve & x axis

• Partitioned areas with alternate signs

• Normalization of area sequence,

- Normalized area - Original area, - Max. area

maxA

AA originalnorm

Increase/decrease in velocity

normA

originalAFig. 9 : Acceleration partition


• To avoid misalignment due to noise

• Compare velocity increment

– Two area sequences compared – Comparison result

• Gesture with min. Value recognized

Imagining curve has mass Obtain center of mass Two curves are aligned to coincide their centers of masses

Subtracting 2 area sequence vectors

nnd

nn

nn

AAAAAAA

AAAAAS

AAAAAS

'2

'21

'1

''1

'3

'2

'12

1........3211

.....

......

,,

21, ssdA


WORK FLOW CHART


MODEL 3:GESTURE RECOGNITION BASED ON SIGN SEQUENCE &

TEMPLATE MATCHING

• Similar to model one

• No encoding of sign sequence in to combinations of -1s & 1s

• Not limited to specific users

Table 2: Gesture codes for model 3


EXPERIMENTAL RESULTS

• ACCURACY Model III > I > II

• PERFORMANCE Model III > I > II

• Model III has an overall mean accuracy of 95.6%

Table 3 : Comparison of gesture recognition accuracy(%) of 3 models


CONCLUSION

Sensor data collection, segmentation & recognition

Sign sequence of gesture is extracted

100’s of data to code of 8 numbers

Code compared with standard patterns


REFERENCES

• WEBSITES ieeexplore.ieee.org/ www.analog.com

MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition , IEEE SENSORS JOURNAL, VOL. 12, NO. 5, MAY 2012.

S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using MEMS motion sensing technology,” in Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008

Template for Presentation _CSE

Documents

gesture circle gesture

acceleration z axis

gesture recognition

dsce fig

z axis real acceleration

sensing system mems

dsce mems accelerometer

axis tick