Kernel-based sparse representation for gesture recognition

Pattern Recognition 46 (2013) 3208–3222

Contents lists available at SciVerse ScienceDirect

Pattern Recognition

0031-32http://d

n CorrSichuan

nn Cor19716, U

E-mcarrillo@kiamile

journal homepage: www.elsevier.com/locate/pr

Kernel-based sparse representation for gesture recognition

Yin Zhou a, Kai Liu a,b,n, Rafael E. Carrillo a, Kenneth E. Barner a,nn, Fouad Kiamilev a

a Department of ECE, University of Delaware, Newark, DE 19716, USAb School of Electrical Engineering and Information, Sichuan University, 610065, China

a r t i c l e i n f o

Article history:Received 12 June 2011Received in revised form28 February 2013Accepted 1 June 2013Available online 15 June 2013

Keywords:Gesture recognitionComputer visionCompressive sensingSparse representationDictionary learning

03/$ - see front matter & 2013 Elsevier Ltd. Ax.doi.org/10.1016/j.patcog.2013.06.007

esponding author at: School of Electrical EUniversity, 610065, China. Tel.: +86 156 0818responding author. Department of ECE, UniverSA. Tel.: +1 302 831 2405.ail addresses: [email protected] (Y. Zhou), kaee.udel.edu (R.E. Carrillo), [email protected] (

[email protected] (F. Kiamilev).

a b s t r a c t

In this paper, we propose a novel sparse representation based framework for classifying complicatedhuman gestures captured as multi-variate time series (MTS). The novel feature extraction strategy,CovSVDK, can overcome the problem of inconsistent lengths among MTS data and is robust to the largevariability within human gestures. Compared with PCA and LDA, the CovSVDK features are more effectivein preserving discriminative information and are more efficient to compute over large-scale MTSdatasets. In addition, we propose a new approach to kernelize sparse representation. Throughkernelization, realized dictionary atoms are more separable for sparse coding algorithms and nonlinearrelationships among data are conveniently transformed into linear relationships in the kernel space,which leads to more effective classification. Finally, the superiority of the proposed framework isdemonstrated through extensive experiments.

& 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Sparse representation has achieved state-of-the-art results inmany fields, such as image compression and denoising [1], facerecognition [2,3], video-based action classification [4], etc. Thesuccess of this technique is partially due to its robustness to noiseand missing data. For example, sparse representation-based clas-sification (SRC) [2] yields impressive results in face recognition byencoding a query face image over the entire set of trainingtemplate images and identifying the label of the query sampleby evaluating which class yields the minimum reconstructionerror. However, little effort has been made to apply this techniqueto classifying multi-variate time series (MTS) data.

Classifying multi-variate time series (MTS) is a challenging task inmany areas, e.g., pattern recognition [5] and computer vision [6]. AnMTS is an m� n matrix, where m is the number of observations onan individual event captured by sensors such as video cameras,position trackers and cybergloves, while n denotes the number ofindependent attributes [7], also known as variables [5,8] or features[9,10]. For each MTS, m is typically varying due to different motiondurations for each instance, while the number of attributes, n, is thesame for all the series since they are recorded by the same set ofdevices. For conventional feature extraction methods, e.g., PCA and

ll rights reserved.

ngineering and Information,5855.sity of Delaware, Newark, DE,

[email protected] (K. Liu),K.E. Barner),

LDA, downsampling and interpolation are usually applied on eachMTS in order to normalize the data length. However, downsamplingmay cause a loss of salient information [5], while interpolation mayinduce distortion to the original data [8].

Gesture MTS data possess both spatial and temporal informa-tion. While spatial information depicts the entire static pattern,temporal information contains the dynamic dependenciesbetween adjacent recordings. Algorithms that exploit chronologi-cal order within time series, e.g., Dynamic Time Warping (DTW)[11,12] and Longest Common Subsequence (LCSS) [13], assumethat similar signals must be recorded in the same order. However,motion order and direction may vary significantly among userspresenting the same gesture. Consequently, such algorithms needto store all possible permutations of each gesture in memory andconduct pair-wise matching during recognition, resulting in exces-sive computation and storage requirements [14]. For example, a2-stroke letter “t” requires 2!� 22 ¼ 8 permutations to represent allpossibilities, while an l-stroke gesture takes l!� 2l permutations.

Notably, real-world gestures and movements, such as humangait and sign language, are performed according to a strict“grammar”. This observation indicates that effectively distinguish-ing complicated spatial patterns is the key to successful recogni-tion, rather than exploiting temporal order [7,5,8]. Motivated bythis observation and reasoning, we consider feature extraction forMTS data ignoring the temporal ordering. More specifically, wegeneralize the capability of SRC to classifying MTS data.

The performance of SRC relies on the quality of the dictionary. Wepropose a novel feature extraction technique, called CovarianceMatrix Singular Value Decomposition for Kernelization (CovSVDK),which possesses three notable merits: CovSVDK is (1) invariant toinconsistent lengths and temporal disorder across MTS data;

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3209

(2) robust to the large variability within human gestures; (3) efficientto compute. In particular, the robustness of the feature extractionstrategy is attributed to the fact that CovSVDK essentially enforces ℓ1minimization algorithms to favor training samples that are consis-tently close to the query sample in every sub-feature space. Moreover,we propose a new approach to kernelize sparse representation. Withthis method, dictionary atoms are more separable for sparse codingalgorithms and nonlinear relationships among data can be conveni-ently transformed into linear relations in kernel space, which leads tomore effective classification. Finally, we evaluate the proposed frame-work over extensive datasets. For the Georgia-Tech HG database, a100% recognition rate is stably achieved; over the High-qualityAustralian Sign Language (HAuslan) database, the recognition accu-racy is greater than 91.2%; for the univariate UCR Time-SeriesRepository, the proposed classifier outperforms competing methodsby achieving the lowest error rate on 10 out of 20 datasets.

The remainder of the paper is organized as follows. First, wegive a brief review of related work and establish the problemformulation in Section 2. In Section 3, the proposed method ispresented. Experiments and comparison with existing methodsare presented in Sections 4 and 5. Finally, we summarize in Section6 and note future directions.

2. Related work and problem formulation

2.1. Related work

Many algorithms have been proposed to measure the similarityamong multi-dimensional time series, e.g. Hidden Markov models(HMMs) [15], DTW [11,12], LCSS [13], and Mixture of Bayes NetworkClassifier [9], among others. Principal components (PCs) basedmethods are, perhaps, the most widely known similarity measurefor multi-attribute time series, with the approach first defined byKrzanowski [16] in 1979. Many subsequent PC efforts focused oncomputing the similarity value using different weighting strategies toaggregate the inner products between PC pairs [5,7,8].

For instance, Li et al. proposed a similarity measure for motionstreams using only the largest singular value and the correspondingsingular vector [7]. In [6], the authors further proposed k WeightedAngular Similarity (kWAS) by considering the k largest singularvalue/vector pairs. Yang and Shahabi [5] proposed a similaritymeasure, called Extended Frobenius norm (Eros), which includedall the singular values by employing a heuristic aggregating func-tion to compute universal weights for all MTS data. The similaritymeasure is a weighted sum of inner products between each pair ofsingular vectors. In practice, however, variance is highly concen-trated in the several largest eigenvalues and the small values aretypically considered as redundancy or noise. Hence, Eros is vulner-able to noise. Yang and Shahabi further extended their approach byusing Eros for Kernel PCA, termed KEros [8].

Recently, some researchers reported the limitation of SRC [2] inclassifying nonlinear data. Zhang et al. [17] proposed the kernel sparserepresentation-based classifier (KSRC) by introducing the kernel trick.However, their approach relies on kernel-based dimensionalityreduction techniques and thus does not offer a direct generalizationto sparse representation in kernel space. Gao et al. [18] proposedkernel sparse representation (KSR). However, the KSR objectivefunction cannot be solved by standard sparse coding algorithms asit requires solving a quadratic programming (QP) problem, which is ofhigher computational complexity than ℓ1 minimization.

2.2. Problem formulation

In a k-label MTS data classification problem, we define thetraining set as T¼ ⋃ki ¼ 1Ti, where Ti ¼ ⋃ni

j ¼ 1ti;j is a subset for the i-th

class with ni samples, and define the query sample as x. Also,denote N¼∑k

i ni as the total number of training samples.There is significant current interest in using SRC [2] to classify

audio, image and video signals. It is therefore desirable to exploreits capability in the field of MTS data classification. To achieve thisgoal, several important issues must be addressed: (1) An effectivefeature extraction method is needed to process large-scale MTSdatasets. The method should be efficient in computation andmemory consumption, and invariant to inconsistent lengths andtemporal disorder across MTS samples. (2) A general formulationof sparse representation suitable for various pattern recognitiontasks is also desired. SRC assumes that training atoms reside on alinear manifold and are distinguishable by ℓ1 minimization algo-rithms. While this premise holds for face images, it does notnecessarily hold for other types of data.

3. Proposed method

This section details methods for effectively extracting MTS datafeatures and present a novel approach to kernelizing sparserepresentation for classification.

3.1. Feature extraction for MTS data

3.1.1. SVD properties of MTS dataFor an m� n MTS t with m observations and n attributes, m is

typically much larger than n and varies across different samples. Inorder to avoid performing SVD on m-varying t, we treat eachattribute (columns in the t) as a random variable and compute thecovariance matrix of t as

Σt ¼ E½tT t�−ET ½t�E½t�; ð1Þwhere E½ � � denotes the mathematical expectation and Σt is offixed dimension n� n (here n≥2). By calculating the Σt of t, wediscard the ordering information and thus overcome the problemof temporal disorder across MTS samples, since each entry in Σt isan inner product between two columns in t that is invariant to therow-switching of t.

Applying SVD to the covariance matrix yields Σt ¼UΛUT , whereU¼ ½u1;…;un� is a singular vector matrix with orthonormalcolumns and Λ¼ diagðρÞ with ρ¼ ½λ1;…; λn�T being a vector withsingular values descendingly sorted. diag is the operator thattransforms ρ into a diagonal matrix by putting entries of ρ alongthe main diagonal in the matrix. Similarly, the covariance matrix

Σp of MTS p can be expressed as Σp ¼VΩVT , where V¼ ½v1;…; vn�and Ω¼ diagðηÞ with η¼ ½ω1;…;ωn�T . Since Σ is positive semi-definite, its SVD is equivalent to eigenvalue decomposition.

If two MTS t and p are similar to each other, ‖Σt−Σp‖F should beclose to zero. In other words, the singular vector ui of Σt shouldresemble vi of Σp in direction and the singular value λi of Σt shouldalso be close to ωi of Σp. Further discussions on the SVD propertiesof MTS can be found in Appendix A.

3.1.2. Simple features for sparse representationFor simplicity, we indicate the i-th training sample as ti.

Applying SVD to the covariance matrix, we get Σti ¼UiΛiUTi , where

Ui ¼ ½u1i ;…;un

i � and Λi ¼ diagðρiÞ with ρi ¼ ½λ1i ;…; λni �T . Note that

uji∈R

n and λji stand for the j-th singular vector (principle compo-nent) and the j-th singular value of ti respectively. We denote

Bj ¼ ½uj1;u

j2;…;uj

N�∈Rn�N as the dictionary containing the j-th

singular vectors extracted from all ti with ‖uji‖2 ¼ 1, for i¼ 1;…;N.

Given a query sample x and corresponding Σx ¼ VΩVT , denotethe j-th singular vector of x as vj and let η¼ ½ω1;…;ωn�T be the

1 Normalization is typically performed to avoid trivial solution and is reason-able in face recognition, since images of a subject under different intensity levelsare still considered to be same-class. In other words, the magnitudes of featurevectors are not considered as discriminative information in face recognition.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223210

vector containing all the singular values in Ω sorted in thedescending order. A simple strategy for classifying x is to treat aparticular vj as the feature of x and employ SRC [2] to identify thefeature by solving

αj ¼ arg minαj

‖αj‖1 subject to Bjαj ¼ vj: ð2Þ

Obtaining αj∈RN , x can be classified by evaluating the class-wise

reconstruction error based on Bj.The above strategy using one singular vector (e.g. the top one)

may work properly with well-separated data. However, real-worldgesture recordings are always vulnerable to noise or large varia-bility among individuals. Therefore it is desirable to take intoaccount several most important singular vectors to improve therobustness of the algorithm. In addition, the discriminative infor-mation within the singular values should also be exploited.

3.1.3. Robust features for sparse representationConsider a robust feature vector constructed by unifying the

top s singular values and the associated singular vectors (s≤n).Suppose that we have obtained αj by solving Eq. (2), for allj¼ 1;…; s. Without violating the equality in the constraint ofEq. (2), we can equivalently rewrite Bjαj ¼ vj as

B̂jα̂ j ¼ λj1

‖ρ1‖2uj1;

λj2‖ρ2‖2

uj2;…;

λjN‖ρN‖2

ujN

" #α̂ j ¼ ωj

‖η‖2vj ð3Þ

where α̂ j ¼Δαj with Δ¼ diagð½ωj‖ρ1‖2=λj1‖η‖2;…;ωj‖ρN‖2=λ

jN‖η‖2�Þ.

Applying the same procedure to each pair of Bj and vj for allj¼ 1;…; s, we get

B̂1α̂1 ¼ ω1

‖η‖2v1

B̂2α̂2 ¼ ω2

‖η‖2v2

…¼…

B̂sα̂s ¼ ωs

‖η‖2vs ð4Þ

Ideally, if x is sufficiently similar to ti, vj should resemble uji, so

should ωj and λji for all j¼ 1;…; s. Therefore, in reconstructing each vj,the uj

i of ti should be coded with large coefficient. In other words, ifeach uj

i of ti contributes most in representing vj of x, ti should besimilar to x. Then, the class to which ti belongs should yield theminimum error in reconstructing x, which indicates that x is of thesame label as ti.

Motivated by this intuition, we enforce each vj of x to berepresented via a universal sparse code α over the correspondingB̂j. By substituting α̂ j with α for all j¼ 1;…; s, Eq. (4) can thus be

simplified as

½B̂1T

; B̂2T

;…; B̂sT �Tα¼ ω1

‖η‖2v1

T;ω2

‖η‖2v2

T;…;

ωs

‖η‖2vsT

� �T; ð5Þ

where ½B̂1T

; B̂2T;…; B̂

sT �T is a vertical concatenation of all the sub-matrices B̂

jand the right hand side is a super-vector by con-

catenating all vj. Thus the classification scheme based on unifyingthe top s pairs of singular values/vectors can be formulated as

α¼ arg minα

‖α‖1 subject to Eq:ð5Þ; ð6Þ

where columns in ½B̂1T

; B̂2T;…; B̂

sT �T are normalized to unitℓ2�norm.

Definition 1 (CovSVDK). Given an MTS t, its covariance matrix isdecomposed as Σt ¼UΛUT by SVD, where U¼ ½u1;…;un� is asingular vector matrix with orthonormal columns andΛ¼ diagðρÞ with ρ¼ ½λ1;…; λn�T is a diagonal matrix with singularvalues descendingly sorted on the main diagonal. The CovSVDK

feature for t is defined as

ϕðtÞ ¼ λ1‖ρ‖2

uT1;

λ2‖ρ‖2

uT2;…;

λs‖ρ‖2

uTs

� �T∈Rsn; ð7Þ

where s subjects to

s¼ arg min∑s

i ¼ 1λi∑n

i ¼ 1λi≥c

� �ð8Þ

for a pre-selected energy threshold, c.

In practice, it is common to empirically set a universal s for allMTS data such that most energy is preserved within the top ssingular values. The name CovSVDK stands for Covariance MatrixSVD for Kernelization.

Definition 2. Given s, define Φ as a collection of features extractedfrom the training set T according to Definition 1, and write Φ as

Φ¼ ½ϕðt1;1Þ;…;ϕðti;1Þ;…;ϕðti;niÞ;…;ϕðtk;nk Þ�∈Rsn�N : ð9Þ

Furthermore, define y¼ ϕðxÞ as the feature of the query sample x.

Discussion: If we define r¼maxðmn;NÞ and denote d as thereduced dimension, PCA is of computational complexity Oðr2dÞwhile CovSVDK is of complexity Oðn2dNÞ. For the cases where m orN is large, Oðr2dÞ⪢Oðn2dNÞ. Thus, CovSVDK is substantially moreefficient than PCA over large-scale datasets or for MTS data withlong durations. More importantly, the memory usage by PCA isproportional to N2 or m2n2 while the memory consumption byCovSVDK is proportional to n2. Hence, CovSVDK is also morememory efficient than PCA.

Revisiting Eq. (5), we can substitute y for ½ðω1=‖η‖2Þv1;

ðω2=‖η‖2Þv2;…; ðωs=‖η‖2Þvs�T∈Rsn and replace ½B̂1T

; B̂2T;…; B̂

sT �T∈Rsn�N with Φ. Finally, the classification scheme based onCovSVDK features can be derived from Eq. (6) as

α¼ arg minα

‖α‖1 subject to Φα¼ y; ð10Þ

where α is the universal sparse code for representing the

ðωi=‖η‖2Þvi over B̂iT

for all i¼ 1;…; s. Limited by space, the robust-ness of CovSVDK features is demonstrated in Appendix B.

3.2. Kernelizing sparse representation for classification

The discrimination capability of SRC relies on the quality of thedictionary. In other words, the atoms associated to different classesmust be distinguishable or separable from the perspective of ℓ1minimization algorithms. In some real-world applications, however,computing the sparse representation over a dictionary of originaltraining features can yield undesirable classification results. One suchexample is the Iris dataset (from UCI machine learning archive). As iscommonly used for analyzing the performance of various classifiers,two features for each sample, regarding pedal length and pedal width,are extracted and formed into a 2D feature vector, as shown inFig. 1(a). The three classes (points in red, green and blue) aredistributed closely along the same radius direction. Obviously, theextracted 2D feature vectors are sufficiently discriminative for tradi-tional classifiers, e.g., k-Nearest-Neighbors (kNN) and Support VectorMachines (SVMs). On the other hand, SRC normalizes training sampleswith unit ℓ2�norm and employs the normalized training samples asdictionary atoms.1 As shown in Fig. 1(b), the atoms are located on theunit circle with severe overlapping in the middle of the point scatter.The atoms within the overlapping region are inseparable and

0 1 2 3 4 5 6 70

0.5

1

1.5

2

2.5

Pedal length

Peda

l wid

th

Training Data ofIris Dataset

SRC loses discriminationabililty

0 0.2 0.4 0.6 0.8 10

0.050.1

0.150.2

0.250.3

0.350.4

0.45

1st dimension

2nd

dim

ensi

on

Dictionary Atomsin SRC

Atoms overlap eachother

−0.2 0 0.2 0.4 0.6 0.8 1 1.20

0.2

0.4

0.6

0.8

1

1st dimension

2nd

dim

ensi

on

Kernelized DictionaryAtoms of SRC

Fig. 1. Training samples and dictionary atoms of SRC. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3211

consequently cause ℓ1 minimization algorithms the confusion inselecting the true atoms. Thus, SRC neglects the magnitude informa-tion and suffers the drawback of losing its discrimination capabilityin classifying data that are distributed along the same radiusdirection [17,19].

We propose the kernelized sparse representation to overcomethis shortcoming of SRC. This is desirable since by kernelizingsparse representation, the classification strategy of SRC can beapplied to general pattern recognition tasks including MTS gesturerecognition, time series classification, etc.

Kernel trick is a widely applied technique in machine learningthat can adapt linear algorithms to nonlinear cases, by mappingtraining features ϕð � Þ from the original space X into some kernelspace F , in which the new kernel features ψð � Þ are more separablefor a certain type of classifiers and the nonlinear relationshipsamong ϕð � Þ∈X can be transformed into linear ones among ψð � Þ∈F .

Let Ψ ¼ ½ψ ðt1;1Þ;…;ψðti;1Þ;…;ψðti;niÞ;…;ψðtk;nk

Þ� be the collectionof training kernel features in F . Given a test sample x, we want tosolve the sparse representation α of ψðxÞ over Ψ . However, this istypically infeasible, as (1) usually the mapping ψ is implicit,meaning that direct evaluation of the fitness term ψðxÞ ¼Ψα isimpossible [17]; (2) F may be of infinite dimension, causing thatthe computational complexity is intractable; (3) even though weknow the mapping explicitly, Ψ TΨ may not be invertible, resultingthat the left inverse does not exist and thus no explicit solution toψðxÞ ¼Ψα is available. To overcome these difficulties, we introducea relaxation to the fitness constraint term as

ψðxÞ0

� �−

Ψ

γI

" #α

����������2

≤ε ð11Þ

where 0∈RN is a zero vector, I∈RN�N is the identity matrix, ε is anarbitrarily small positive constant representing the error tolerance, γis a small positive constant. Satisfying Eq. (11) is equivalent tominimizing the ridge regression problem LðαÞ ¼ ‖ψðxÞ−Ψα‖22 þ γαTα.Setting the gradient of LðαÞ with respect to α equal to zero, thesolution space of α is obtained as

Ψ TψðxÞ ¼ ðΨ TΨ þ γIÞα ð12Þ

where Ψ Tψ ðxÞ is an N � 1 vector and Ψ TΨ is an N � N positive semi-definite matrix. Regularized by γ, ðΨ TΨ þ γIÞ is invertible, yieldingthat α is the global minimizer to LðαÞ. In other words, enablingEq. (12) is equivalent to satisfying Eq. (11). Thus, we can employEq. (12) as the fitness constraint in sparse coding.2

2 Note that the proposed relaxation to fitness constraint (Eqs. (11) and (12)) is ageneral strategy and is applicable to kernelizing other sparse coding algorithms,such as Orthogonal Matching Pursuit (OMP), but in this paper we only focus on ℓ1minimization algorithms.

To improve the efficiency in ℓ1 minimization and to ensure thesolution to be sparse, a random matrix P∈Rd�N obeying Gaussianor Bernoulli distribution (we use Gaussian here) is often employedto project vector Ψ TψðxÞ and columns in ðΨ TΨ þ γIÞ into somed-dimensional random subspace, where d5N.

Define the K¼ Ψ TΨ as a Gram matrix, with elementsKi;j ¼ kðϕðtiÞ; ϕðtjÞÞ, where kð�; �Þ is a valid kernel function. By

denoting ~y ¼ Ψ TψðxÞ ¼ kð�; xÞ∈RN and substituting K for Ψ TΨ inthe new fitness constraint Eq. (12), the kernelized sparse repre-sentation under random projection P is formulated as

α¼ arg minα

‖α‖1 subject to PðKþ γIÞα¼ P ~y : ð13Þ

From Eq. (13), we can see that the linear relationship betweenkernel features ψðxÞ and columns in Ψ has been depicted entirelyin terms of the linear combination between the kernel functionvalues in vector ~y and the corresponding ones in matrix K. For thepurpose of effectively classifying MTS gestures and time seriesdata, we further propose two kernel functions based on theCovSVDK features.

Proposition 1 (Kernel function). Let t and p be two samples and letϕðtÞ and ϕðpÞ be their extracted feature vectors. The proposed kernelfunction is defined as

kðϕðtÞ;ϕðpÞÞ ¼ expfkLðϕðtÞ;ϕðpÞÞg ¼ ψðtÞTψðpÞ ð14Þwhere ψðtÞ∈F and ψðpÞ∈F are kernel features for t and p, via someimplicit nonlinear mapping ψ . In particular, for MTS data, ϕðtÞ andϕðpÞ are extracted according to Definition 1 and the kernel functionkLð�; �Þ can be written as

kLðϕðtÞ;ϕðpÞÞ ¼ ϕðtÞTϕðpÞ ¼ ∑s

i ¼ 1

λiωi

‖ρ‖2‖η‖2

� �uTi vi: ð15Þ

Note that kernel features ψð � Þ∈F are of infinite dimension. Byworking directly on the kernel function however, we can implicitlyexploit the kernel space of high, or even infinite dimension,without the need of knowing mapping ψ . By using the proposedkernel function kð�; �Þ, the atoms embedded in a 2D randomsubspace for the Iris dataset are separable for ℓ1 minimizationalgorithms, as shown in Fig. 1(c).

By incorporating the classification rule of SRC into Eq. (13), weobtain the newly proposed classifier, called Kernelized SRC, whichshall be discussed in the following two sections.

3.3. Training

Building a discriminative dictionary is critical to the effective-ness of sparse representation based classifiers. Given a training

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of randomly chosen features

Rec

ogni

tion

rate

Dim of random subspace = 20Dim of random subspace = 40Dim of random subspace = 60Dim of random subspace = 80Without random projection

1 4 7 10 13 16 19 22 25 28

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimension of the random subspace

Rec

ogni

tion

rate

Fig. 2. Recognition rate for the Georgia-Tech HG database. (a) 15-class problem recognition rate versus selected features (markers) under various random projections. Thehorizontal axis represents the number of randomly chosen features, ranging from 2 to 22. The curves in different colors represents recognition rate over 5 different randomsubspaces. (b) 15-class problem recognition rate versus dimensions of random subspace; 22 features (markers) are employed. (For interpretation of the references to color inthis figure legend, the reader is referred to the web version of this article.)

3 Published by the Computational Perception Laboratory at Gatech at http://www.cc.gatech.edu/cpl/projects/hid/.

4 Published by UCI KDD at http://kdd.ics.uci.edu/summary.data.date.html

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223212

set T, we now describe how to construct such a dictionary viakernel trick based on specific feature extraction methods.To elaborate, we first use median filter to preprocess each sample(in the noisy case). Then we loop through all training samples tocompute the features. For MTS data, the CovSVDK feature isextracted individually from each training sample. For the case ofunivariate time series data, we simply employ each raw time seriesas a feature vector, since CovSVDK is effective only when n≥2.Next, we construct a dictionary as the regularized kernel matrixKþ γI. Finally, we may employ a random matrix P to improve theefficiency in classification. The whole training process is summar-ized in Algorithm 1.

Algorithm 1. Kernelized SRC: Training.

Require: Training set T1: Preprocess each training sample with median filter(optional)2: for i¼1 to k do3: for j¼1 to ni do4: Feature extraction for each ti;j-ϕðti;jÞ

(for MTS data, ϕðti;jÞ is extracted according to Definition 1)5: end for6: end for7: Compute K according to Proposition 18: Construct dictionary as Kþ γI9: Secure sparsity in the solution vector by employing P fordimensionality reduction (optional)10: return P and PðK þ γIÞ

3.4. Classification

In this section, we discuss how to classify a query sample usingthe proposed Kernelized SRC. Having x as a test sample, we firstpreprocess it with the same technique as in training and extract itsfeature as y¼ ϕðxÞ. Then based on the kernel function defined inProposition 1, we have ~y ¼ kð�; xÞ ¼ ½kðϕðt1Þ;ϕðxÞÞ;…; kðϕðtNÞ;ϕðxÞÞ�T∈RN . Next, random projection can be performed to reducedimensionality. Then, we find the sparse representation α of ~y overPðKþ γIÞ by solving the optimization problem Eq. (13), which iscalled Basis Pursuit Denoising (BPD) [20]. Notice that the sparsecoefficients, α, can be computed by other fast iterative algorithms,such as Orthogonal Matching Pursuit [21] or Compressive SamplingMatching Pursuit [22]. Experimental results reported in the

following sections are based on the ℓ1 Magic implementationof BPD [23]. Finally, we identify x as class i based on the decisionrule as

i¼ arg mini∈f1;…;kg

‖P ~y−PðKþ γIÞδiðαÞ‖2; ð16Þ

where δiðαÞ ¼ ½0;…; αi;1;…; αi;ni ;…;0�. To cope with unbalancedclasses, an alternative decision rule i¼ arg mini∈f1;…;kg‖P ~y−PðKþγIÞδiðαÞ‖2=‖δiðαÞ‖1 can be employed. The classification procedure issummarized in Algorithm 2.

Algorithm 2. Kernelized SRC: Classification.

Require: Test sample x, random matrix P and dictionaryPðKþ γIÞ1: Preprocess test sample with median filter (optional)2: Feature extraction for x-y¼ ϕðxÞ according to Definition 13: Based on the kernel function defined in Proposition 1,compute~y ¼ kð�;xÞ ¼ kðϕðt1Þ;ϕðxÞÞ;…; kðϕðtNÞ;ϕðxÞÞ

T4: Random subspace embedding via P (optional)5: Find the sparse coefficient vector α by solving Eq. (13)6: i¼ arg mini∈f1;…;kg‖P ~y−PðK þ γIÞδiðαÞ‖27: return i

4. Experiments on classifying real-world MTS data

In this section, we conduct experiments to demonstrate thepromising performance of the proposed framework, i.e., CovSVDK+ Kernelized SRC, over three online public-access databases, i.e.,the Georgian-Tech Human Gait (Georgia-Tech HG) database,3

Australian Sign Language (Auslan) database4 and High-qualityAustralian Sign Language (HAuslan) database.4 The Georgia-TechHG database was obtained via 12 video cameras; the Auslan wasgenerated by Powergloves; and the HAuslan was generated by two5DT gloves and two position trackers. To verify the effectiveness ofthe proposed CovSVDK feature, we use the linear kernel kLð�; �Þ forall experiments in this section. Feature vector ϕð � Þ for each MTS isextracted according to Definition 1. For each particular database,the parameter s is manually selected and is consistent for all MTS

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3213

data within the database. As in [2], atoms in PðKþ γIÞ are normal-ized to unit ℓ2�norm prior to ℓ1 minimization. γ is set to 0.001.

We evaluate and compare the proposed CovSVDK, with PrincipleComponent Analysis (PCA) and Linear Discriminant Analysis (LDA).For PCA and LDA, all MTS data are interpolated or downsampled tothe average length, in each database. We compare the proposedclassifier Kernelized SRC with two popular classifiers, i.e., K-Nearest-Neighbor (KNN) with k¼3, Support Vector Machines (SVM) andwith the coding strategy by computing the least square solution toEq. (12), termed LS. For Kernelized SRC and LS, the decision rule isEq. (16). For KNN and SVM, columns in Φ are employed as trainingdata and ϕðxÞ is used as the test sample. The SVM toolbox can befound at [24]. As shown in the following, our method consistentlyachieves high performance over these databases.

4.1. Georgia-Tech HG database

The Georgia-Tech HG database, used for human identificationfrom a distance, is a collection of human gaits from 15 subjects.Samples of subjects were captured by cameras at 4 differentcontrolled speeds [9]. Every subject was required to walk 9 timesat every controlled speed and finally, 36 samples were obtained forevery subject. A sample is a time series of gaits with varyinglength. By means of 22 markers on the subject, a gait is defined by66 attributes (variables), i.e., the 3-D coordinates of those markers[25,26]. The evaluation uses all the 540 samples in the database.

6 18 30 42 54 660

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Raw Measurement attributes

Rec

ogni

tion

rate

CovSVDK + SVMCovSVDK + KNNCovSVDK + LSCovSVDK + Kernelized SRC

Fig. 3. Recognition rate for various methods over the Georgia-Tech HG database.

1 5 9 13 17 21 25 29

00.10.20.30.40.50.60.70.80.9

1

Dimension of Feature Subspace

Rec

ogni

tion

rate

1 2 3 4 5 6

00.10.20.30.40.50.60.70.80.9

1

Dimension of Fe

Rec

ogni

tion

rate

PCA feature + SVMPCA feature + KNNPCA feature + LSPCA feature + Kernelized SRC

LLLL

Fig. 4. Recognition rate on the Georgia-Tech HG database. (a) PCA feature, (b) LDA featurare fed to four classifiers, i.e., SVM, KNN, LS, the proposed Kernelized SRC.

Among the 36 samples per subject, 30 samples are randomlycollected into the training set while the remaining 6 samples areused for testing.

By transforming the kernel matrix into a low dimensionalrandom subspace, we can reduce the computation cost of ℓ1minimization. In order to evaluate the effectiveness of randomprojection, we randomly select parts of the overall 22 markers andset the parameter s¼5 uniformly, such that 5 singular value/vectorpairs are extracted by CovSVDK for each MTS. Fig. 2(a) indicatesthat the proposed approach can achieve 100% recognition ratewhen a random subspace is of only 20 dimensions and only 11markers are utilized. Hence, in the following experiments over thisdatabase, kernel matrices are projected onto a random subspacewith dimension 20 to improve computation efficiency.

Remark. It is worthy to point out that, for ℓ1 minimizers, thedimensionality reduction induced by random projection is not arequisite. The purpose of embedding the dictionary atoms intosome low-dimensional subspace is two-fold: (1) speed-up ℓ1minimization; (2) enforce the dictionary to be overcomplete suchthat the solution tends to be sparse. The first concern is desiredfrom a practical efficiency perspective while the second concern ispreferred by the decision rule (Eq. (16)) so as to secure satisfactoryrecognition rate. We can see from Fig. 2(b) that the recognitionrate increases as the dimension of the random subspace becomeshigher. For completeness, we also evaluate the proposed approachover the Georgia-Tech HG database without performing dimen-sionality reduction. Fig. 2(a) illustrates that the accuracy obtainedwithout dimensionality reduction is similar to those with dimen-sionality reduction.

To evaluate the proposed framework in a more challengingscenario, we downsample the raw gesture data into 1/5 of itsoriginal length and utilize only part of the overall 66 attributes. Asshown in Fig. 3, our method robustly achieves 98.9% recognition,leading SVM by approximately 10% in accuracy.

As shown in Fig. 4, at 9, 4, and 1 dimension(s) of the featuresubspace respectively, PCA, LDA and CovSVDK achieve 100%recognition rate. Therefore, compared with PCA and LDA, theproposed CovSVDK is more effective in preserving discriminativeinformation for classification. Finally, Table 1 shows that inclassifying MTS data, the proposed linear kernel function kLð�; �Þsignificantly outperforms three other popular kernel functions, i.e.,exponential, polynomial and Gaussian kernel functions.

4.2. Australian sign language (Auslan) database

Contributed by 5 individual signers, the Auslan database con-tains 95 one-hand signs. 70 samples were collected for each signand a sample is comprised of varying-length time series for a

7 8 9 10 11 12 13 14

ature Subspace

1 2 3 4 5 6

00.10.20.30.40.50.60.70.80.9

1

Dimension of Feature Subspace

Rec

ogni

tion

rate

DA feature + SVMDA feature + KNNDA feature + LSDA feature + Kernelized SRC

CovSVDK + SVMCovSVDK + KNNCovSVDK + LSCovSVDK + Kernelized SRC

e, and (c) CovSVDK feature (proposed method). All three feature extraction methods

Table 1Comparison among different kernel functions over the Georgia-Tech HG database.

Database Proposed kL Exponential Poly. (d¼3) Gaussian

Gait (%) 100 92.2 85.6 80.4

Table 2Binary classification comparison among various methods over the Auslan database.

Method Training set Test set Recognition rate (%)

Proposed 36 4 96.3MixCML [9] 39 1 95.5a

DTW [11] 39 1 88a

a Recognition rates are cited from [9].

Table 3Binary classification result over the Auslan database for various selection ofattributes.

Method Selected attributes Recognition rate (%)

Proposed 1th–4th, 7th–10th 96.3%1th–6th 94.51th–4th 96.37th–10th 70.01th–3th 96.3

5 This is due to the fact that Kernelized SRC uses the simplest linear kernelwhile SVM employs the more advanced RBF kernel.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223214

single hand gesture. There are 15 attributes or features for eachgesture, i.e., the x, y and z coordinates of the palm, the angles(roll, pitch and yaw) of the palm, the bend values of the 5 fingersand 4 additional setting values. Over this database, we conductcomparative study by evaluating the proposed approach (CovSVDK +Kernelized SRC) against several state-of-the-art algorithms, i.e.,discriminative mixture learning (MixCML [9]), Dynamic TimeWarping (DTW) [11], Fourier Descriptors [27] and SRC [2]. Recogni-tion rates are cited from literature for the first three methods.Results for SRC are reported based on our own implementation.

In the first experiment over the Auslan database, we consider abinary classification task. With the same experiment setup as [9],we form a subset by using 10 signs and choose, from the 15attributes, 8 attributes, namely the x, y and z coordinates of thepalm, the roll angle of the palm, the bend values of the fingers ofthumb, fore, index and ring.

For each of the 10 signs, i.e., “eat”, “exit”, “forget”, “give”, “hello”,“know”, “love”, “no”, “sorry” and “yes”, we select approximately4 samples from each signer. Conducting 10-fold cross-validationyields a training set of 36 samples (18 per sign) and a test set of4 samples. The proposed framework is compared with MixCML [9]and DTW [11], and the results are listed in Table 2. For complete-ness, the proposed method is further examined by performingbinary classification over various selection of attributes. Theresults are summarized in Table 3. Consistent with the argumentmade by Kim and Pavlovic [9], our observation also reveals thatthe 7th–10th attributes are less discriminative than others as theyonly provide the finger flexion information.

In literature, we notice that this database has been widelyapplied to evaluate spatial trajectory recognition algorithms. In thesecond experiment, for fair comparison, we only keep 3 attributes,i.e., x, y and z coordinates. Fig. 5 gives some examples for 8 signs.Using the same CovSVDK features, we first compare two classi-fiers, i.e., Kernelized SRC and SRC based on 10-fold cross-valida-tion. Then keeping the experiment setup consistent as in [27], theproposed approach (CovSVDK + Kernelized SRC) is compared with

DTW [11] and Fourier Descriptor [27] based on 2-fold cross-validation. Classification results for aforementioned methods aresummarized in Table 4, which indicates that the proposed algo-rithm is competitive among these advanced trajectory recognitionalgorithms.

The effectiveness of Kernelized SRC is illustrated in Fig. 6, inwhich, for better visualization, 15 samples per sign are utilized fortraining while the remaining 5 samples are for testing. The 2/3Dmanifolds are obtained by projecting the dictionaries (with andwithout the kernel trick) into random subspace. Clearly, withkernel trick, samples from different classes are more separablethan those without the kernel trick, which reveals that theproposed classifier is more robust than SRC [2] when dealing withcluttered data.

4.3. High-quality Australian sign language (HAuslan) database

The HAuslan database consists of 95 two-hand signs. Comparedwith the Auslan database, the number of samples per sign isreduced to 27 and the number of attributes is increased to 22,(11 attributes for each hand). The 11 attributes for one hand arethe same as those in Auslan database excluding the 4 settingvalues.

First, to illustrate the capability of our method in classifyinglarge-scale databases, all 95 sign classes are used. Since theHAuslan database contains much more classes but fewer samplesper class than previous two databases, 24 randomly selectedsamples are assigned to training set for each sign, while theremaining 3 samples are collected into the test set. Note that thekernel matrix contributed by all training samples is of size2280�2280, to which performing ℓ1 minimization is computa-tionally expensive. For efficient classification, we employ randomprojection to reduce the row dimension of the kernel matrix to 40,which is just 1.8% of its original size. In addition, considering thatthe subtle differences among some signs, we set c¼ 99:9% so as toinvolve sufficient gesture details to enable effective classification.To improve robustness and remove outlier atoms from thedictionary, we apply a refinement process to the dictionary byonly preserving the atoms with large reconstruction coefficients,based on the solution to Eq. (13). Then, the newly formed sub-dictionary is fed to the classifier. The recognition rates of theproposed framework (CovSVDK + Kernelized SRC) are presented inTable 5 and in Fig. 7.

Next, we compare CovSVDK + Kernelized SRC with variouscombinations of feature extraction strategies and classifiers. ForCovSVDK, we set the parameter smax ¼ 6 and for PCA, we set theenergy preservation ratio cmax ¼ 99:9%, which results in a maximal30 features. The maximal number of linear features for LDA is 21.Fig. 8 shows that although Kernelized SRC using PCA and LDAfeatures yields inferior performance to SVM,5 when workingjointly with CovSVDK, Kernelized SRC outperforms other combi-nations of features and classifiers. This result confirms the effec-tiveness of the proposed framework. The highest recognition ratesand the corresponding dimensions of feature space for variousmethods are summarized in Table 6. As shown in Table 7, inclassifying MTS data, the proposed kernel function kLð�; �Þ againsignificantly outperforms three other widely used kernel func-tions, i.e., exponential, polynomial and Gaussian kernel functions.

Finally, a comparison among state-of-the-art methods in the25-label classification problem is given in Table 8, which furthervalidates the superiority of the proposed method.

−0.6−0.4

−0.20

0.2

−0.20

0.20.4

0.60.8

−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

−0.3−0.2

−0.10

0.1

−0.5

0

0.5

1−0.05

−0.04

−0.03

−0.02

−0.01

0

0.01

−0.3 −0.25−0.2 −0.15

−0.1 −0.05 0 0.05

−0.5

0

0.5

1−0.03

−0.02

−0.01

0

0.01

0.02

0.03

0.04

−0.8−0.6

−0.4−0.2

00.2

−0.10

0.10.2

0.30.4

−0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

−0.8−0.6

−0.4−0.2

00.2

−0.10

0.10.2

0.30.4

−0.08−0.1

−0.06

−0.04

−0.02

0

0.02

−0.4−0.3

−0.2−0.1

00.1

00.1

0.20.3

0.4−0.2

−0.15

−0.1

−0.05

0

−0.25−0.2 −0.15−0.1 −0.05 0 0.05 0.1

−0.5

0

0.5

1−0.035

−0.03

−0.025

−0.02

−0.015

−0.01

−0.005

0

−0.2 −0.15−0.1 −0.05 0 0.05 0.1 0.15

−0.20

0.20.4

0.60.8

−0.04

−0.03

−0.02

−0.01

0

0.01

Fig. 5. 3D trajectories for 8 signs. (a) Eat, (b) Exit, (c) Forget, (d) Give (e) Hello, (f) Know, (g) Love, and (h) No.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3215

4.4. Evaluating the robustness

In this section, we evaluate the robustness of the proposedframework by employing the Sparsity Concentration Index (SCI)

[2] to detect outliers. The SCI is defined as [2]

SCIðαÞ ¼ k �maxi‖δiðαÞ‖1=‖α‖1−1k−1

; ð17Þ

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223216

where α is the solution to Eq. (13) and δiðαÞ is the characteristicfunction defined in Eq. (16). If a test sample can be entirelyexpressed by the training samples from only a single class, thenSCIðαÞ ¼ 1; while, in the other extreme, if the coefficients in αspread evenly over the classes, then SCIðαÞ ¼ 0. The intuition lies inthe fact that, for a test sample belonging to a certain class in thetraining set, the large sparse coefficients should be mostly con-centrated on the same-class training samples and therefore yield

Table 4Multi-class classification comparison among various methods over the Auslandatabase. Proposed 1 is based on 10-fold cross-validation; For proposed 2, the datapool is divided into 2 folds, i.e., one fold for training and the other fold for test,according to [27].

Method Train set: test set Classes (%)

2 3 4 8

Proposed 1 0:9 : 0:1 96.3 93.3 90.6 80.0SRC [2] 0:9 : 0:1 78.5 73.3 70.9 63.0Proposed 2 0:5 : 0:5 96.0 92.7 88.0 75.4DTW [11] 0:5 : 0:5 89.8a N/A 83.8a 75.9a

Fourier descriptor [27] 0:5 : 0:5 82.1a N/A 63.7a 52.3a

a Recognition rates are cited from [27].

0.1 0.15 0.2 0.25 0.3 0.350.955

0.96

0.965

0.97

0.975

0.98

0.985

0.99

0.995

1Sign "Eat" training dataSign "Exit" training dataSign "Forget" training dataSign "Eat" test dataSign "Exit" test dataSign "Forget" test data

−

−

−

−

0.40.5

0.60.7

0.740.760.780.80.820.840.860.88

−0.7

−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0Sign "Eat" training dataSign "Exit" training dataSign "Forget" training dataSign "Give" training dataSign "Eat" test dataSign "Exit" test dataSign "Forget" test dataSign "Give" test data

−

Fig. 6. Illustrations of manifolds in multi-class classification tasks. Top row: the 3-labemanifold without the kernel trick, (c) 3D manifold with the kernel trick, and (d) 3D ma

an SCI that approaches 1. On the other hand, if the test sample isan irrelevant outlier, then its sparse coefficients should spreadalmost evenly across the whole training set and yield an SCI closeto 0. Thus, the outlier detection criterion [2] is established, bysetting a threshold τ∈ð0;1Þ, where a test sample is rejected asoutlier if SCIðαÞoτ.

We verify the robustness of the proposed method over theGeorgia-Tech HG and the HAuslan databases. As recommended in[2], we incorporate approximately half of all the classes into thetraining set but keep the test set containing samples from all theclasses. Thus almost half of the test set are considered as irrelevantoutliers with respect to the dictionary. For the two databases, thenumber of classes employed in the training set are 8 and 48respectively. We test the performance of the proposed algorithm(CovSVDK + Kernelized SRC) by ranging τ from 0 to 1 with 0.01

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

0.8

0.6

0.4

0.2

0

0.2

0.4Sign "Eat" training dataSign "Exit" training dataSign "Forget" training dataSign "Eat" test dataSign "Exit" test dataSign "Forget" test data

0.20.4

0.60.8

1

−1−0.5

00.5

1

0.1

0

0.1

0.2

0.3

0.4

Sign "Eat" training dataSign "Exit" training dataSign "Forget" training dataSign "Give" training dataSign "Eat" test dataSign "Exit" test dataSign "Forget" test dataSign "Give" test data

l task; bottom row: the 4-label task. (a) 2D manifold with the kernel trick, (b) 2Dnifold without the kernel trick.

Table 5Recognition rate on the HAuslan database. The dimension of random subspace isfixed at 40 for all the classification tasks.

Classes:samples 20:540 25:675 40:1080 95:2565

Recognition rate (%) 98.2 97.6 94.3 91.2

5 10 15 20 25 30 35 40 45 50 55

0.4

0.5

0.6

0.7

0.8

0.9

1

Dimension of Random Subspace

Rec

ogni

tion

rate

No. of Classes = 20No. of Classes = 25No. of Classes = 40No. of Classes = 95

Fig. 7. Recognition rate for the HAuslan database.

2 6 10 14 18 22 26 30

00.10.20.30.40.50.60.70.80.9

1

Dimension of Feature Subspace

Rec

ogni

tion

rate

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

00.10.20.30.40.50.60.70.80.9

1

Dimension of Feature Subspace

Rec

ogni

tion

rate

1 2 3 4 5 6

00.10.20.30.40.50.60.70.80.9

1

Dimension of Feature Subspace

Rec

ogni

tion

rate

PCA feature + SVMPCA feature + KNNPCA feature + LSPCA feature + Kernelized SRC

LDA feature + SVMLDA feature + KNNLDA feature + LSLDA feature + Kernelized SRC

CovSVDK + SVMCovSVDK + KNNCovSVDK + LSCovSVDK + Kernelized SRC

Fig. 8. Recognition rate over the HAuslan database. (a) PCA feature, (b) LDA feature, and (c) CovSVDK feature (proposed method). All three feature extraction methods are fedto four classifiers, i.e., SVM, KNN, LS, the proposed Kernelized SRC.

Table 6Summary of recognition performance on the HAuslan database.

Methods Proposed PCA+SVM LDA+SVM LDA+KNN

Features 6 28 18 18Accuracy (%) 91.2 83.4 90.0 90.4

Table 7Comparison among different kernel functions over the HAuslan database.

Database Proposed kL Exponential Poly. (d¼3) Gaussian

HAuslan (%) 91.2 76 75.8 78.9

Table 8Comparison of recognition rate among various methods over the HAuslan database.

Method Proposed Li [7] 2dSVD [28] SegSVD [29]

Accuracy (%) 97.6 89.0a 95.0a 93.9a

a Recognition rates are cited from references.

6 To avoid the similarity values out of range, a normalizing ϕð � Þ to unitℓ2�norm or dividing the matrix entries by N is needed. We choose the formerstrategy in this work.

7 The information regarding classic machine learning algorithms is summar-ized in http://www.cs.ucr.edu/�eamonn/time_series_data/WekaOnTimeSeries.xls.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3217

step size. The resulting Receiver Operator Characteristic (ROC)curves, (Fig. 9), indicate that: (1) the proposed CovSVDK outper-forms classical LDA in outlier detection; (2) Kernelized SRCdemonstrates improved robustness compared to SRC; and (3) the

Area Under Curve (AUC) of the proposed framework exceeds theAUC of other listed approaches.

5. Experiments on classifying real-world univariate time seriesdata

In this section, we evaluate the proposed classifier KernelizedSRC with nonlinear kernel function kð�; �Þ over 20 datasets (data1)from UCR Time-Series Repository [30]. Raw time series are directlytreated as feature vectors ϕð � Þ without using CovSVDK, which iseffective only when n≥2.6 The regularization parameter γ is set to0.001. All columns in PðKþ γIÞ are normalized to unit ℓ2�normprior to sparse coding. The dictionary employed is the kernelmatrix with compression rates fd=N¼ 0:10;0:25;0:50;noneginduced by random projection, where none means no dimension-ality reduction. The best result from the four cases is reported.

We compare Kernelized SRC with state-of-the-art time seriesclassifiers, i.e.,1NN-Best Warping Window DTW [12], Time Seriesbased on a Bag-of-Features representation (TSBF) [31], as well as7 classic classifiers.7 The error rates of all methods are listed inTable 9, from which we can see that Kernelized SRC leadsother algorithms by yielding the lowest error rate in 7 out of the

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Positive Rate

True

Pos

itive

Rat

e

CovSVDK + Kernelized SRC AUC = 0.991.CovSVDK + SRC AUC = 0.972.LDA + SRC AUC = 0.821.LDA + LS AUC = 0.624.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Positive Rate

True

Pos

itive

Rat

e

CovSVDK + Kernelized SRC AUC = 0.915.CovSVDK + SRC AUC = 0.892.LDA + SRC AUC = 0.781.LDA + LS AUC = 0.688.

Fig. 9. ROC curves for outlier detection over the Georgia-Tech HG and the HAuslan databases. (a) The Georgia-Tech HG database and (b) the HAuslan database. CovSVDmeans feature extraction following Definitions 1 and 2, before applying the kernel trick.

Table 9Classification results on UCR time-series repository. Note that DTW⋆ [12] means 1NN-Best Warping Window DTW and TSBFn [31] represents Time Series based on a Bag-of-Features representation with the optimal parameter setting z¼0.25. Results for compared methods are cited from references.

Knn (%) NB (%) C45 (%) MLP (%) RandForest (%) LMT (%) SVM (%) DTW⋆ [12] (%) TSBFn [31] (%) Kernelized SRC (%)

50words 35.60 43.74 58.24 33.63 44.84 43.08 35.38 24.20 19.10 32.16Adiac 40.66 43.22 46.80 25.06 42.20 27.88 56.01 39.10 28.60 37.34Beef 40.00 50.00 43.33 26.67 50.00 20.00 33.33 46.70 35.00 14.44CBF 15.00 10.33 32.67 14.67 16.44 23.00 12.33 0.40 0.50 12.89Coffee 25.00 32.14 42.86 3.57 25.00 0.00 3.57 17.90 0.40 0.00ECG200 11.00 23.00 28.00 16.00 19.00 18.00 19.00 12.00 13.80 9.00FaceAll 31.36 30.83 44.97 17.57 39.05 24.26 28.17 19.20 21.70 11.08FaceFour 12.50 15.91 28.41 12.50 21.59 22.73 11.36 11.40 3.80 18.18Fish 21.71 33.14 40.00 16.00 20.57 18.29 14.86 16.00 7.10 12.57Gun point 8.00 21.33 22.67 6.67 10.67 20.67 20.00 8.70 1.10 4.00Lighting2 19.67 32.79 37.70 26.23 21.31 36.07 27.87 13.10 24.90 22.95Lighting7 36.99 35.62 45.21 35.62 43.84 35.62 28.77 28.80 30.70 37.54OliveOil 23.33 23.33 26.67 13.33 13.33 16.67 13.33 16.70 11.30 3.33OSULeaf 45.45 62.81 63.22 55.37 58.26 50.83 56.20 38.40 23.30 42.56SwedishLeaf 20.32 14.56 34.40 13.44 22.24 17.44 15.84 15.70 8.90 11.52Synthetic control 12.00 4.00 19.00 8.67 14.00 8.00 7.67 1.70 1.90 2.67Trace 18.00 20.00 26.00 23.00 19.00 24.00 27.00 1.00 2.00 13.00Two patterns 9.40 54.33 34.88 10.35 27.50 16.78 17.80 0.15 0.10 12.92Wafer 0.60 29.17 1.80 3.72 0.68 1.91 4.04 0.50 0.40 0.38Yoga 16.70 45.77 30.10 25.50 22.13 28.13 36.93 15.50 16.00 14.81

DTW Accuracy

Ker

nel S

RC

Acc

urac

y

0.5 0.6 0.7 0.8 0.9 10.5

0.6

0.7

0.8

0.9

1

Fig. 10. Accuracy scatter plot between Kernelized SRC and 1NN-Best Warping Window DTW [30]. Each dot represents a dataset. Dots above the diagonal mean that KernelizedSRC is better than 1NN-Best Warping Window DTW and vice versa. The farther away a dot is from the diagonal, the greater the accuracy improvement achieved [13].

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223218

SRC Accuracy

Ker

nel S

RC

Acc

urac

y

0.5 0.6 0.7 0.8 0.9 10.5

0.6

0.7

0.8

0.9

1

Expected Accuracy Gain

Act

ual A

ccur

acy

Gai

n

0.5 1 1.5 2 2.5 30.5

1

1.5

2

2.5

3

TP

TN

FN

FP

Two Patterns

Fig. 11. Comparison between Kernelized SRC and SRC. (a) accuracy scatter plot; (b) expected accuracy gain versus actual accuracy gain. Note that regions marked as TP/TNrepresent we correctly predict Kernelized SRC is better/worse than SRC; region FN means that we predict Kernelized SRC is worse than SRC but the fact is the opposite;region FP means that we predict Kernelized SRC is better than SRC but the fact is the opposite. Practically, only FP is the truly bad case [32].

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

0

20

40

60

80

100

120

140

160

180

200

Indices of singular values

Sing

ular

val

ues

Alive 1Alive 2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

0

20

40

60

80

100

120

140

160

180

200

Indices of singular values

Sing

ular

val

ues

Alive 1All 1

0 1 2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Indices of pairwise group

Cos

ine

valu

es b

etw

een

two

dire

ctio

ns

1st principla component2nd principal component3rd principal component

Fig. A1. Discriminative properties of SVD for MTS data from the HAuslan database. (a) Pairwise comparison of singular values between two MTS data, both representing sign“Alive”; (b) pairwise comparison of singular values between two MTS data, respectively representing the signs “Alive” and “All”; (c) comparison of the directions of thesingular vector pairs. Group 1 is the comparison between twoMTS data from the same class “Alive” and group 2 is the comparison between two MTS data from classes “Alive”and “All” respectively.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3219

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223220

20 datasets. In particular, we visualize the accuracy scatter plotbetween Kernelized SRC and 1NN-Best Warping Window DTW[12], which is considered one of the best time series classifiers. Asshown in Fig. 10, the proposed classifier slightly outperforms 1NN-Best Warping Window DTW in 11 out of 20 datasets.

In addition, to fully justify the effectiveness of the proposedkernelization strategy, we test SRC over the 20 datasets and compareit with Kernelized SRC by visualizing the accuracy scatter plot. Fig. 11(a) shows that using kernel trick significantly improves the classifica-tion performance, as Kernelized SRC outperforms SRC in 19 out of 20datasets. Moreover, a classifier is useful only if we can predict ahead oftime on which datasets it will generate higher accuracy. We thereforeperform further experiments to verify the reliability of Kernelized SRCby evaluating the expected accuracy gain versus the actual accuracygain [32]. To acquire the expected accuracy gain, we conduct leave-one-out cross-validation within the training set for both algorithms.The gain is calculated as [32] g ¼ Accuracy Kernelized SRC=Accuracy SRC. As depicted in Fig. 11(b), 19 out of the 20 dots are inregion TP with the remaining 1 in region TN, which indicates that theperformance of Kernelized SRC is completely predictable over the 20datasets. From the same figure, we also observe that a remarkable 20%or even higher performance increase compared to SRC is achieved viakernelization over a majority of the datasets. The impressive resultsvalidate that the proposed Kernelized SRC is very effective for timeseries classification.

6. Conclusion and future work

In this paper, we propose a novel sparse representation basedframework for classifying complicated human gestures captured

−1 −0.8

−0.6

−0.4

−0.2 0 0.2 0.4

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1st Dim

2nd

Dim

1st 2D Singular Vector

Class 1Class 2Class 3Class 4Class 5Class 6Class 7Class 8Class 9Class 10Test 1Test 2

Test 1

Test 2

−

2nd

Dim

0.7

0.8

0.9

1

1st Dim

2nd

Dim

Test 1 RegionZoom−in

0.7

0.8

0.9

1st Dim

2nd

Dim

Test 2 RegionZoom−in

−0.5

−0.45 −0

.4−0

.35 −0.3

−0.7

−0.6

−0.5

Fig. B1. (a) and (b) are 100 2D training sub-features (Eq. (3)) obtained from Australian Sigonly the x; y attributes; (c)–(f) are region zoom-in of test point 1 (test 1) and test poin

as multi-variate time series (MTS). First, we propose a featureextraction strategy, called CovSVDK, which is invariant to incon-sistent lengths and temporal disorder across MTS data, robust tovariability within human gestures, and efficient to compute. Inaddition, we propose a new approach to kernelize sparse repre-sentation by introducing a relaxation to the fitness constraint. Thistechnique is generic and can be applied to kernelizing other sparsecoding algorithms. Using this technique, we derive a classifiercalled Kernelized SRC, which is very effective in classifying MTSdata and univariate time series as shown in the experiments.

In our future work, the proposed approach will be combined witha multi-layer structure that can model complicated temporal varia-tions to further improve the recognition performance. The perfor-mance of incorporating nonlinear kernel function into Kernelized SRCfor classifying gesture MTS data will also be extensively investigated.

Conflict of Interest

The authors have no conflict of interest.

Acknowledgments

This material is based upon work supported by the NationalScience Foundation under Grant no. 0812458.

Appendix A. SVD properties of MTS data

As shown in Fig. A1(a), the singular values between two same-class MTS data, both standing for the sign “Alive” from HAuslan

00.0

5 0.1 0.15 0.2 0.2

5 0.3 0.35 0.4

−0.1

0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

1st Dim

2nd 2D Singular Vector

Class 1Class 2Class 3Class 4Class 5Class 6Class 7Class 8Class 9Class 10Test 1Test 2

Test 1

Test 2

−0.1

0

0.1

1st Dim

2nd

Dim

Test 1 RegionZoom−in

Consistentneighbor

0.02

0.04

0.06

0.08

0.01

0.015 0.0

20.0

25

−0.05

0

0.05

1st Dim

2nd

Dim

Test 2 RegionZoom−in

Consistentneighbors

n Language (Auslan) database by performing SVD over each training MTS regardingt 2 (test 2) in the 1st and 2nd 2D sub-feature space.

0 10 20 30 40 50 60 70 80 90 100

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60 70 80 90 100

0

0.2

0.4

0.6

0.8

0 10 20 30 40 50 60 70 80 90 100

0

0.002

0.004

0.006

0.008

0.01

0.012

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3x 10−3

0 10 20 30 40 50 60 70 80 90 100

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 10 20 30 40 50 60 70 80 90 100

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Fig. B2. Sparse codes computed based on different strategies. (a)–(d) are sparse codes solved based on Eq. (2) for each test point over the corresponding dictionary in each2D feature space. (e) and (f) are sparse codes solved based on Eq. (10).

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–3222 3221

database, resemble to each other correspondingly and shrink tozero quickly as the index growing. For the two MTS data pertinentto different classes (sign “Alive” and sign “All” from HAuslandatabase), the singular values differ significantly from each other,as shown in Fig. A1(b). To measure the resemblance in directionbetween two singular vectors, we can simply compute the cosinevalue of the acute angle between them defined as

cos θ¼ j⟨ui; vi⟩j; ðA:1Þwhere θ∈½0; π=2� and ⟨; ⟩ is the inner product operator. As illu-strated Fig. A1(c), similarity value between singular vectors(principle component) of two same-class MTS data is close to 1,meaning that their directions are similar. On the other hand, if thetwo MTS data are associated to different classes, the similarityvalue between singular vectors is significantly smaller than 1.Therefore, the singular values and singular vectors obtained viaSVD provide discriminative information for classifying MTS data.

Appendix B. Effectiveness of CovSVDK features

The robustness of the proposed CovSVDK feature extraction isillustrated in Figs. B1 and B2. Here Australian Sign Language(Auslan) database is employed by only keeping x; y information,such that each MTS contains two attributes, i.e., n¼2. ApplyingSVD to each MTS yields two singular vectors, each with dimension2. Fig. B1(a) and (b) are 2D sub-features (i.e., ðλ1=‖ρ‖2ÞuT

1 andðλ2=‖ρ‖2ÞuT

2) extracted from 100 training MTS data correspondingto the 1st and the 2nd singular vector respectively. Two testsamples (Test 1 and Test 2), all associated to class 1, are selected.For Test 1, the same-class neighbor is consistently near it in both

sub-feature spaces (Fig. B1(c) and (e)). For Test 2, three same-classneighbors are consistently close to it in both sub-feature spaces(Fig. B1(d) and (f)), while the similarities between Test 2 andsamples from other classes vary significantly throughout the twosub-feature spaces. Using CovSVDK features, SRC can leverage theconsistent closeness between the test sample and its same-classneighbors across different sub-feature spaces by computing auniversal sparse code. Thus, Test 1 and Test 2 are correctlyclassified into class 1, as shown in Fig. B2(e) and (f). On the otherhand, the classification scheme using one single singular vector,i.e., Eq. (2), causes misclassification, as shown in Fig. B2(a)–(d).

References

[1] M. Aharon, M. Elad, A. Bruckstein, K-svd: an algorithm for designing over-complete dictionaries for sparse representation, IEEE Transactions on SignalProcessing 54 (11) (2006) 4311–4322.

[2] J. Wright, A. Yang, A. Ganesh, S. Sastry, Y. Ma, Robust face recognition viasparse representation, IEEE Transactions on Pattern Analysis and MachineIntelligence 31 (2) (2009) 210–227.

[3] Q. Zhang, B. Li, Discriminative k-svd for dictionary learning in face recognition,in: IEEE Conference on Computer Vision and Pattern Recognition, 2010,pp. 2691–2698.

[4] Y. Li, C. Fermuller, Y. Aloimonos, H. Ji, Learning shift-invariant sparserepresentation of actions, IEEE Conference on Computer Vision and PatternRecognition, 2010, pp. 2630–2637.

[5] K. Yang, C. Shahabi, A PCA-based similarity measure for multivariate timeseries, in: Proceedings of the 2nd ACM International Workshop on MultimediaDatabases (MMDB' 04), 2004, pp. 65–74.

[6] C. Li, S. Q. Zheng, B. Prabhakaran, Segmentation and recognition of motionstreams by similarity search, ACM Transactions on Multimedia Computing,Communications and Applications 3 (3), http://dx.doi.org/10.1145/1236471.1236475.

Y. Zhou et al. / Pattern Recognition 46 (2013) 3208–32223222

[7] C. Li, P. Zhai, S. Zheng, B. Prabhakaran, Segmentation and recognition of multi-attribute motion sequences, in: Proceedings of the ACM Multimedia Con-ference, 2004, pp. 836–843.

[8] K. Yang, C. Shahabi, A PCA-based kernel for kernel PCA on multivariate timeseries, in: Proceedings of ICDM 2005 Workshop on Temporal Data Mining:Algorithms, Theory and Applications, 2005, pp. 149–156.

[9] M. Kim, V. Pavlovic, Discriminative learning of mixture of Bayesian networkclassifiers for sequence classification, in: IEEE Conference on Computer Visionand Pattern Recognition, 2006, pp. 268–275.

[10] Y. Yuan, K.E. Barner, Hybrid feature selection for gesture recognition usingsupport vector machines, in: IEEE Conference on ICASSP, 2008, pp. 1941–1944.

[11] H. Sakoe, S. Chiba, Dynamic programming algorithm optimization for spokenword recognition, IEEE Transactions on Acoustics, Speech and Signal Proces-sing 26 (1) (1978) 43–49.

[12] C.A. Ratanamahatana, E.J. Keogh, Making time-series classification moreaccurate using learned constraints, in: SDM'04, 2004.

[13] M. Vlachos, M. Hadjieleftheriou, D. Gunopulos, E. Keogh, Indexing multi-dimensional time-series with support for multiple distance measures, ACMSIGMOD (2003) 216–225.

[14] A.L. Vatavu, R.-D.J. Wobbrock, Gestures as point clouds: a $ p recognizer foruser interface prototypes, in: International Conference on Multimodal Inter-action, 2012.

[15] F. Bashir, A. Khokhar, D. Schonfeld, Object trajectory-based activity classifica-tion and recognition using hidden Markov models, IEEE Transactions on ImageProcessing 16 (7) (2007) 1912–1919.

[16] W.J. Krzanowski, Between-groups comparison of principal components, Jour-nal of Acoustical Society of America 74 (367) (1979) 703–707.

[17] L. Zhang, W.-D. Zhou, P.-C. Chang, J. Liu, Z. Yan, T. Wang, F.-Z. Li, Kernel sparserepresentation-based classifier, IEEE Transactions on Signal Processing. 60 (4)(2012) 1684–1695.

[18] S. Gao, I. Tsang, L.-T. Chia, Kernel sparse representation for image classificationand face recognition, in: European Conference on Computer Vision 2010(ECCV 2010).

[19] Y. Zhou, J. Gao, K.E. Barner, An enhanced sparse representation strategy forsignal classification, in: Proceedings, SPIE Defense, Security, and Sensing, 2012.

[20] E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signalreconstruction from highly incomplete frequency information, IEEE Transac-tions on Information Theory 52 (2) (2006) 489–509.

[21] J. Tropp, A. Gilbert, Signal recovery from random measurements via orthogo-nal matching pursuit, IEEE Transactions on Information Theory 53 (12) (2007)4655–4666.

[22] D. Needell, J.A. Tropp, Cosamp: iterative signal recovery from incomplete andinaccurate samples, Communications of ACM 53 (12) (2010) 93–100.

[23] E. Candès, J. Romberg, l1-magic: Recovery of sparse signals via convexprogramming. /http://users.ece.gatech.edu/justin/l1magic/downloads/l1magic.pdfS.

[24] C.-C. Chang, C.-J. Lin, Libsvm: a library for support vector machines, 2001.Available at ⟨http://www.csie.ntu.edu.tw/�cjlin/libsvm⟩.

[25] R. Tanawongsuwan, A. Bobick, Characteristics of Time–Distance Gait Para-meters Across Speeds, GVU Technical Report.

[26] R. Tanawongsuwan, A. Bobick, Performance analysis of time-distance gaitparameters under different speeds, in: 4th International Conference on Audioand Video Based Biometric Person Authentication, 2003.

[27] S. Wu, Y. Li, On signature invariants for effective motion trajectory recognition,International Journal of Robotics Research 27 (8) (2008) 895–917.

[28] X. Weng, J. Shen, Classification of multivariate time series using two-dimensional singular value decomposition, Knowledge-Based Systems 21 (7)(2008) 535–539.

[29] J. Liu, M. Kavakli, Hand gesture recognition based on segmented singular valuedecomposition, in: Knowledge-Based and Intelligent Information and Engi-neering Systems, vol. 6277, 2010, pp. 214–223.

[30] E. Keogh, Q. Zhu, B. Hu, H.Y.X. Xi, L. Wei, R.C.A. Duizer, The UCR Time SeriesClassification/Clustering, 2011. Available at ⟨http://www.cs.ucr.edu/eamonn/time_series_data/⟩.

[31] M.G. Baydogan, G. Runger, E. Tuv, A bag-of-features framework to classifytime series, IEEE Transactions on Pattern Analysis and Machine Intelligence,http://dx.doi.org/10.1109/TPAMI.2013.72, in press.

[32] G.E.A.P.A. Batista, X. Wang, E.J. Keogh, A complexity-invariant distancemeasure for time series, in: SIAM Conference on Data Mining, 2011.

Yin Zhou received the B.S. degree in electrical engineering from Beijing Jiaotong University, China in 2009. He is currently a Ph.D. student in the ECE department, Universityof Delaware. His research interests include machine learning, computer vision, sparse signal processing and stereo vision.

Kai Liu received the B.S. and the M.S. degrees in computer science from the Sichuan University, China in 1996 and 2001, and the Ph.D. degree in electrical engineering fromthe University of Kentucky, USA in 2010, respectively. He is currently a postdoctoral researcher at the Information Access Lab in the University of Delaware since September2010. His main research interests include computer/machine vision, active/passive stereo vision, image processing and gesture recognition.

Rafael E. Carrillo (S'07) received the B.S.E.E. and the M.S.E.E. degrees (with honors) from the Javeriana University, Bogotá, Colombia, in 2003 and 2006 respectively. He was aresearch assistant and a visiting lecturer from 2003 to 2006 at the Javeriana University. Currently, he is a Ph.D. candidate at the Department of Electrical and ComputerEngineering at the University of Delaware, Newark, DE. His research interests include signal and image processing, visual speech processing, compressive sensing, inverseproblems, and robust, nonlinear and statistical signal processing. Mr. Carrillo was the recipient of the “Mejor trabajo de grado” award, given to outstanding master thesis atthe Javeriana University, in 2006, the University of Delaware Competitive Graduate Student Fellowship in 2007 and the Signal Processing and Communications Faculty Awardin 2010 (award is presented to an outstanding graduate student in this research area).

Kenneth E. Barner (S'84-M'92-SM'00) received the B.S.E.E. degree (magna cum laude) from Lehigh University, Bethlehem, Pennsylvania, in 1987. He received the M.S.E.E. andthe Ph.D. degrees from the University of Delaware, Newark, Delaware, in 1989 and 1992, respectively. For his dissertation “Permutation Filters: A Group Theoretic Class ofNon–Linear Filters,” Dr. Barner received the Allan P. Colburn Prize in Mathematical Sciences and Engineering for the most outstanding doctoral dissertation in the engineeringand mathematical disciplines. Dr. Barner was the duPont Teaching Fellow and a Visiting Lecturer at the University of Delaware in 1991 and 1992, respectively. From 1993 to1997 Dr. Barner was an Assistant Research Professor in the Department of Electrical and Computer Engineering at the University of Delaware and a Research Engineer at theduPont Hospital for Children. He is currently Professor and Chairman in the Department of Electrical and Computer Engineering at the University of Delaware, and a SeniorMember of the IEEE. Dr. Barner is the recipient of a 1999 NSF CAREER award. He was the co-chair of the 2001 IEEE–EURASIP Nonlinear Signal and Image Processing (NSIP)Workshop and a guest editor for a special issue of the EURASIP Journal of Applied Signal Processing on Nonlinear Signal and Image Processing. Dr. Barner is a member of theNonlinear Signal and Image Processing Board and is co-editor of the book Nonlinear Signal and Image Processing: Theory, Methods, and Applications, CRC Press, 2004. Dr. Barnerwas the Technical Program co-Chair for ICASSP 2005 and is currently serving on the IEEE Signal Processing Theory and Methods (SPTM) technical committee, previously servedon the IEEE Bio–Imaging and Signal Processing (BISP) technical committee, and is currently a member of the IEEE Delaware Bay Section Executive Committee. Dr. Barner hasserved as an associate editor of the IEEE Transactions on Signal Processing, the IEEE Transaction on Neural Systems and Rehabilitation Engineering, and the IEEE Signal ProcessingMagazine. Dr. Barner is currently the Editor in Chief of the journal Advances in Human-Computer Interaction, an associate editor for IEEE Signal Processing Letters, a member ofthe Editorial Board of the EURASIP Journal of Applied Signal Processing, and served as a guest editor for that journal on the Super–Resolution Enhancement of Digital Video andEmpirical Mode Decomposition and the Hilbert-Huang Transform special issues. His research interests include signal and image processing, robust signal processing, nonlinearsystems, sensor networks and consensus systems, compressive sensing, human-computer interaction, haptic and tactile methods, and universal access. Dr. Barner is amember of Tau Beta Pi, Eta Kappa Nu, and Phi Sigma Kappa.

Fouad Kiamilev is a professor in the Department Electrical and Computer Engineering at the University of Delaware where he directs a research group called CVORG (whichstands for CMOS VLSI Optimization Research Group). Fouad's main mission is to train students to become successful participants in the 21st century global economy. Since1997, he has advised 14 Ph.D. students and 20 M.S. students. His graduates are employed by leading academic and industrial organizations in the United States. Fouad'sresearch group specializes in custom hardware design for special applications. As a hobby the group likes to tackle the security problems of today from a hardwareperspective.