Real-Time Feature Extraction and Classification of Prehensile EMG Signals Master Thesis Christopher Miller Supervisor: Marko Vuskovic Department of Computer.

Real-Time Feature Extraction and Classification of Prehensile EMG Signals

Master Thesis

Christopher Miller

Supervisor: Marko Vuskovic

Department of Computer ScienceSan Diego State University

March 24, 2008

2 Copyright © 2008 by Christopher Miller

Agenda

• Introduction• Electromyography (EMG) Signals• EMG Signal Processing• Classification• Experimental Results• Implementation• Conclusion• Future Work• Questions


Introduction(1 of 2)

• Numerous technological advances in prosthetic hands

• Greater degrees of freedom

• Continue to function as pincers


Introduction(2 of 2)

The purpose of this thesis was to implement a program that could perform real-time feature extraction and classification of prehensile EMG signals for the following grasps:


Electromyography (EMG) Signals:Myoelectric Energy Detection

• Motor units control groups of muscle fibers

• Brain recruits motor units to innervate muscles for movement

• Myoelectric energy produced as motor units activate

• Surface electrodes detect myoelectric energy


Electromyography (EMG) Signals:EMG Amplification

• SENIAM Recommends– Pre-gelled Ag/Ag-Cl– Bipolar– 0.8” inter-electrode distance– 0.4” wide

• Surface electrodes– Pre-gelled Ag/Ag-Cl– 1” inter-electrode distance– 0.875” wide

• EMG amplification device (Saksit Siriprayoonsak, 2005)

– 4 bipolar channels– 1 reference channel


Electromyography (EMG) Signals:Muscle Anatomy


Electromyography (EMG) Signals:Mounted EMG Amplifier


EMG Signal Processing:System Flowchart

Prosthetic Hand

Controller

EMG Amplification

Device

A/D Converter

Classifier

Signal Processing

Time Sample Extraction

Feature Extraction

Transformation


EMG Signal Processing:Onset of Movement Detection

• Lidierth (1986)• Hodges & Bui (1996)• Bonato et al. (1998)• Staude et al. (2001) – AGLRamp, AGLRstep


EMG Signal Processing:Bonato Method

hgkt

yyg

ka

kkk

:5,... 3, 1,min

,1 22

120


EMG Signal Processing:Feature Extraction Methods

• Mean Absolute Value (Hudgins et al., 1993)

• Mean Absolute Value Slope (Hudgins et al., 1993)

• Willison Amplitude (Willison, 1964)

• Zero Crossings (Hudgins et al., 1993)

• Slope Sign Changes (Farry et al., 1996)

• Waveform Length (Farry et al., 1996)

• Simple Square-Integral

• Amplitude of the First Burst (Vuskovic et al., 2002)

• Multiple Time Windows (Du et al., 2003)

• Short-Time Fourier Transform

• Wavelet Transform

• Wavelet Packet Transform

• Spectral Moments (Vuskovic et al., 2005)


EMG Signal Processing:Feature Extraction Comparison

Method Parameters 400 ms 300 ms 200 ms MAV 96.11% 95.56% 93.89% MAVSLP I = 3 83.89% 81.67% 73.89% VAR 90.56% 90.00% 86.67% WAMP H = 29 98.33% 95.56% 92.22% Waveform Length 98.89% 98.89% 95.00% Zero-Crossings H = 28 96.11% 94.44% 83.89% Slope Sign Changes H = 100 96.67% 96.67% 86.67% Squared Integral 90.56% 90.00% 86.67% Spectral Moments m = 1

K = 11 flag = 0

92.78% 89.44% 82.22%


EMG Signal Processing:Feature Extraction Methods Employed (1 of 3)

Spectral Moments:

I-coefficients, calculated in advance:

1

1

π2][)(N

Nk

kfjSS ekCfP

1

0

)1( ..., 2, 1, ],[][1

][kN

iSS Nkkisis

NkC

K

kmSSmSSm kIkCICM

1

)(][2)0(]0[

)1(2

1)0(

1 m

Imm

4322210 )2(

22)1(3)( ,

)2(

)1()( ,

)2(

1)1()( ,0)(

k

kkI

kkI

kkIkI

kkkk

... 3, 2, 1, ,

)2(

6)1(2)(

44

kk

kkI

kk

kIm

k

mkI m

mk

mm 21

21121

)2(2)(

K

kmSS

mmm kIkC

MIMR

1

0 )(][2

)0(2



Parameter Range Purpose k 0..N

(N is sample size) Time-lag parameter in I-coefficient and autocorrelation function

m 0..4 Highest moment index to use in feature vector (i.e. m = 2 means that M0, M1, and M2 are included)

Ts 0..N Size of time sample to use for moment calculation Flag 0..2 0 – use straight moments

1 – use M0 and reduced moments for higher values of m 2 – use strictly reduced moments

Spectral Moment Parameters



Waveform Length:

N

kkk xxWL

21


Classification:Methods

• Artificial Neural Networks (McCulloch-Pitts, 1943)• ARTMAP Networks (Grossberg et al., 1976)• Mahalanobis-Distance Based ARTMAP Network

(Xu et al., 2003)• Maximum Likelihood Estimation (Fisher, 1912)• Mahalanobis Distance (Mahalanobis, 1936)


Classification:Methods Employed

• Maximum Likelihood Estimation (MLE)

• Mahalanobis Distance

)()(

2

1exp

)2(

1)|( 1

21 CC

TC

C

xSxS

xp

)()()( 1 xSxxMD CCT

C

])([)cov( TXXc XXEXS


Experimental Results:Impact of Log Transformation


Experimental Results:Box-Cox Transformation

• Box-Cox (1964):

0 ,log

0 ,1

)(

y

yy

0 ),log(

0 ,1)(

12

11

2)(

1

y

yy


Experimental Results:λ Optimization

• Mahalanobis-distance Classifier using Spectral Moments– Flag = 0

– K = 11

– Ts = 250


Experimental Results:Moment Optimization

• Classifiers using Spectral Moments– Flag = 0

– K = 11

– Ts = 400

– Lam = 0.06


Experimental Results:Time Sample Optimization

• Mahalanobis-distance Classifier using Spectral Moments– M = 2

– Flag = 0

– K = 11

– Lam = 0.06


Experimental Results:Channel Reduction (1 of 2)


Experimental Results:Channel Reduction (2 of 2)

• Mahalanobis-distance Classifier using Spectral Moments– M = 2

– Flag = 0

– K = 11

– Lam = 0.06

– 3 channels


Experimental Results:Feature Comparison (1 of 2)

Waveform Length -

- Spectral Moments


Experimental Results:Feature Comparison (2 of 2)

MAH Classifier Performance at 264 ms Using Various Feature Vectors

Hybrid Percent Used for Testing

Spectral Moments

Waveform Length M=0 M=1 M=2

10%: 99.34% 97.79% 98.54% 98.82% 98.51% 20%: 99.20% 97.83% 98.31% 98.27% 97.37% 30%: 98.84% 97.89% 97.90% 97.46% 95.45% 40%: 97.96% 97.70% 97.56% 96.20% 89.33% 50%: 96.30% 97.59% 96.92% 93.36% 26.07% 60%: 26.87% 97.37% 95.77% 25.88% N/A 70%: N/A 96.93% 89.73% N/A N/A 80%: N/A 95.10% 25.78% N/A N/A

Average of 50% and below

98.33% 97.76% 97.85% 96.82% 81.35%


Experimental Results:Hybrid Approach


Experimental Results:Spectral Moments

• Best classification rate: 97.5%

• Optimal time sample: 378 ms


Experimental Results:Waveform Length

• Best classification rate: 95%

• Optimal time sample: 300 ms


Experimental Results:Waveform Length Online Testing

• Cross-validation– 95% at 300 ms

• Online validation– Sphere: 22/25 (88%)

– Cylinder: 25/25 (100%)

– Precision: 21/25 (84%)

– Lateral: 17/25 (68%)

– Total: 85%


Experimental Results:Spectral Moments Online Testing

• Cross-validation– 89.4% at 355 ms

• Online validation– Sphere: 24/25 (96%)

– Cylinder: 25/25 (100%)

– Precision: 25/25 (100%)

– Lateral: 19/25 (76%)

– Total: 93%


Implementation:Program Files

Name Type Purpose

EMGProgGUIFC.m GUI support functions for program EMGProgGUIFC.fig GUI front-end for program Train.m Function to generate classifier parameters S, C, N GetClass.m Function to classify data using current classifier LOOValidation.m Leave One Out Validation procedure used for training BonatoOnset.m Function to determine onset of movement in recorded data BonatoRealTime.m Function to determine onset of movement in real-time Features.m Function to create feature vector using spectral moments Moments.m Function to compute spectral moments of time signal GenerateI.m Function to generate I coefficients for moment calculation FeaturesWL.m Function to create feature vector using waveform length feature FeaturesMAV.m Function to create feature vector from mean absolute value BoxCox.m Function to perform Box-Cox transformation dflt_train.mat MAT file containing classifier based on research data EMGDLLmfile.m

MATLAB

DAQ functions MATLAB prototype file EMGDLL.dll Dynamic Link Library (DLL) for DAQ card DAQ_Functions.h National Instruments DAQ functions header DAQ_Functions.c

C

National Instruments DAQ functions implementation


Implementation:Main Screen


Implementation:Training (1 of 3)



• Leave-one-out validation from 200-400 ms in 10 ms intervals

• Optimal time explored at 1 ms intervals

• Both features scored based on classification rate and time sample required

• Highest scoring feature selected and system trained



• May load training from files

• Training recordings automatically stored with classifier

• Training recordings saved to EMGRecordings folder


Implementation:Real-Time Grasp Classification

Rest Position

Start Grasp

Hold Grasp

Onset of movement detected

Grasp classified

Grasp tension below threshold

Grasp tension above threshold

Collect grasp data

Onset of movement not

detected


Implementation:Real-Time Grasp Classification Screenshot

Implementation:Real-Time Grasp Classification Video



Conclusion

• Feasible approach demonstrated for real-time classification of prehensile EMG signals

• More natural approach reduces mental and physical demands on operator

• Only 3 EMG channels are necessary to classify the 4 grasps• Waveform Length proved to be valuable for small training sets• Spectral Moments enabled better performance (93% online)

with larger training sets and optimized parameters– K = 11– m = 2– flag = 0– λ = 0.06– Ts optimized for training set


Future Work:Additional Grasps

• Previous research explored classifying 6 grasps, which included small and large versions of balls and cylinders

• Two-phased approach to classification is more likely to succeed


Future Work:Full Control

• Channel 4 of current device can be employed for grasp control


Future Work:Online Learning

• Real-Time program should be written in multi-threading capable language

• Online learning capability with feedback from prosthetic hand

• Mahalanobis-distance based ARTMAP network suggested

EMG Classifier Library

(MATLAB DLL with C Interface)

Deployed Real-Time Program

(C++ or Java)

Robotic Hand

Controller (C/C++)

EMG Device Drivers

(C)

Real-Time EMG Classifier GUI

(MATLAB)

Train classifier

Upload classifier

Read EMG Data

Classify EMG Data

Update grasp

Questions


Backup Slides


Differential Amplification

Real-Time Feature Extraction and Classification of Prehensile EMG Signals Master Thesis Christopher Miller Supervisor: Marko Vuskovic Department of Computer.

Documents

mounted emg amplifier

aglrstep slide

emg amplification seniam

myoelectric energy slide

muscle anatomy slide

system flowchart slide

reference channel slide

feature extraction methods