LNCS 3851 - Modelling the Effect of View Angle Variation on
Appearance-Based Gait RecognitionModelling the Effect of View Angle
Variation on Appearance-Based Gait Recognition
Shiqi Yu, Daoliang Tan, and Tieniu Tan
National Laboratory of Pattern Recognition, Institute of
Automation, Chinese Academy of Sciences,
P.O.Box 2728, Beijing, 100080, China {sqyu, dltan,
tnt}@nlpr.ia.ac.cn
Abstract. In recent years, many gait recognition algorithms have
been devel- oped, but most of them depend on a specific view angle.
However, view an- gle variation is a significant factor among those
that affect gait recognition per- formance. It is important to find
the relationship between the performance and the view angle. In
this paper, we discuss the effect of view angle variation on
appearance-based gait recognition performance. A multi-view gait
database (124 subjects and 11 view directions) is created for our
research. We propose two mod- els, a geometrical one and a
mathematical one, to model the effect of view angle variation on
appearance-based gait recognition. These models will be valuable
for designing robust gait recognition systems.
1 Introduction
Gait, as an attractive biometric for human identification at a
distance, has received an increasing interest from researchers in
the computer vision community. The study by Murray et al. [1] is
supportive of the uniqueness of gait for a person. More
importantly, gait has the advantages of being non-contact,
non-invasive and easily acquired at dis- tance in contrast with
other biometrics.
Among the factors which affect gait recognition performance, such
as view angle, clothing, shoe type, carrying condition and surface
type [2], view angle variation is a significant one since for a
given gait recognition system, it is impossible to expect all the
subjects to walk in a particular direction. Furthermore, most
appearance-based gait recognition algorithms depend upon a specific
view angle. Some researchers in computer vision have been devoting
their efforts to designing view-invariant and multi- view
algorithms. Johnson et al. [3] propose a multi-view algorithm,
which recovers static body parameters of subjects and uses these
view-invariant parameters to recognize people. Kale et al. [4] use
a sophisticated method to eliminate the effect from view angle
change. They synthesize the side view from another arbitrary view
using a single camera through the perspective projection model or
the optical flow-based structure from motion equations.
Although many gait recognition algorithms for human identification
have been pro- posed and developed over the past years, most of
them are view-dependent, which will limit their practical
applications. In addition, there are two remaining open
problems.
P.J. Narayanan et al. (Eds.): ACCV 2006, LNCS 3851, pp. 807–816,
2006. c© Springer-Verlag Berlin Heidelberg 2006
808 S. Yu, D. Tan, and T. Tan
One is which view is the most suitable for gait recognition and why
it is. Kale et al [4] declare that the side view is the best choice
in practice, but no theoretical results, for the time being, have
been given to prove that. Another open problem is how view angle
variation affects the performance of gait recognition. Intuitively,
the greater the angle between the gallery (training) set and the
probe (test) set is, the worse the recognition performance.
However, there are not been experimental or theoretical results on
the relationship between gait recognition performance and view
angles. It is obvious that the answers to the above questions are
significant for designing robust gait recognition systems.
This paper proposes two models, a geometrical one and a
mathematical one, in an attempt to address these two questions. The
purpose of this paper is to investigate and analyze the effect of
view angle on the performance of appearance-based gait recogni-
tion.
The remainder of this paper is organized as follows. Section 2
presents our definition of performance function. In Section 3, we
discuss a multi-view gait database. Then, Section 4 introduces our
experiments and gives experimental results. Two models are given in
Section 5. Finally, this paper is concluded in Section 6.
2 Performance Evaluation Function
In our experiments, the correct classification rate (CCR) is used
to evaluate the per- formance of gait recognition. Suppose, without
the loss of generality, that the angle between the view direction
of gallery set and the walking direction is θg , and the angle
constituted by the view direction of probe set and the walking
direction is θp. Obviously the CCR is a function of variables θg
and θp.
CCR = f(θg, θp) θg ∈ [0, 360), θp ∈ [0, 360) (1)
If we can get the analytic expression of the function f(θg, θp),
then how θg and θp
affect the recognition performance can be solved with ease.
Discovering the expression of f(θg, θp), however, can not be easily
achieved. It is impossible to precisely obtain the value of f(θg,
θp) at any point in space P = [0, 360) × [0, 360) by way of experi-
mental methods as P is a continuous space. A sophisticated way to
solve this problem is to compute the value of f(θg, θp) through
experiments at a discrete and limited subset of P. The subset can
be P = {θ, 2θ, · · · , 360} × {θ, 2θ, · · · , 360}, and θ is a
small angle.
As mentioned above, the video data ought to be collected from view
angles ranging from θ to 360 at an increment θ. When the camera is
far from the subject, the sil- houette taken from the left hand
side of the subject is, from the perspective of geometry, basically
similar to that from the right hand side. Therefore, the video data
just need to be collected from only one side (the left side in this
paper). The video data in our ex- periments is collected at view
angles {0, θ, 2θ, · · · , 180}, and θ = 18. The CCR in the discrete
subset {0, 1, 2, · · · , 180
θ } × {0, 1, 2, · · · , 180 θ } can be obtained
by experiments and be formulated as Equation(2).
Modelling the Effect of View Angle Variation on Appearance-Based
Gait Recognition 809
CCR = F (n, k) = f(n · θ, k · θ)
n = 0, 1, 2, · · · , ⌊180
θ
⌋ (2)
θ
⌋
An algebraic formula f(θg, θp), which is an approximation to f(θg,
θp) and satisfies f(θg, θp) ≈ f(θg, θp), can be acquired by data
fitting and interpolation to F (n, k). A simple yet useful and
reasonable model f(θg, θp) is presented in Section 6 on the basis
of numerically analyzing the experimental results. The CCR at
arbitrary θg and θp can be predicted or estimated with this model.
It is easy to imagine that this work has a great practical
meaning.
3 A Multi-view Gait Database
To analyze the impact of view angle changes on gait recognition
performance, a multi- view gait database is needed. In addition to
consisting of a great number of subjects, the database should be
composed of the gait data collected from many view angles. The
minimum angle interval ought to be relatively small.
For the purpose of developing gait recognition algorithms, a
variety of gait databases have been created by many research units,
such as USF [2], Soton [5], CASIA [6],
Computer 1
Computer 3
Computer 2/controller
Fig. 1. The schematic diagram of gait data collection system
Fig. 2. Sample frames from 11 view angles
810 S. Yu, D. Tan, and T. Tan
UMD, etc. The existing gait databases are either not large enough,
or only captured from few view angles. These gait databases do not
fulfill the requirements of view angle variation research. We
develop a gait data collection system for creating a multi- view
gait database which meets the requirements of view angle variation
research. 11 cameras were used to capture gait videos as
illustrated in Fig. 1.
All the subjects are asked to walk naturally on the concrete ground
along a straight line in an indoor environment. The videos can be
simultaneously captured by 11 cam- eras from different view
directions. At last we successfully collect 124 subjects’ gait data
(94 males and 30 females). The view angle θ between the view
direction and the walking direction takes on the values of 0, 18,
36, · · · , and 180, as delineated in Fig. 1. Each subject walks
along the straight line 10 times (6 for normal walking, 2 for
walking in a coat and 2 for walking with a bag), and 11 video
sequences are captured each time. Thus, there are 110 sequences for
each subject, and a total of 110 × 124 = 13640 video sequences in
our database. All the video sequences have the same resolution of
320 × 240 pixels. Some sample frames from 11 cameras are shown in
Fig. 2.
Our database comprises those factors affecting gait recognition:
view angles (11 views), clothing (with or without in a coat), and
carrying condition (with or without a bag). Only view angles is
studied here, though other factors are interesting to study
too.
4 Gait Feature Extraction
There are many appearance-based gait features in the literature.
Most of them are ex- tracted from human silhouettes or outer
contours. Here we choose one typical feature from each category.
One is gait energy image (GEI), and it is extracted from human
silhouettes. GEI is introduced in [7], which is the average of all
silhouettes in a video sequence. The other is key Fourier
descriptors (KFDs), and it is extracted from human outer contours.
KFD method is proposed by Yu et al. [8], which is the key compo-
nent of Fourier descriptors computed from human contours. Finally,
we use the nearest neighbor classifier to perform
classification.
4.1 Silhouette Segmentation
Given a fixed camera, the human silhouette can be extracted by
background subtraction and thresholding. We take advantage of the
method given in [9] to segment human sil- houette from image
sequences. The sizes of the silhouettes we extracted are not
unique, and the silhouettes need to be normalized to the same
size.
4.2 GEI Feature Extraction
The gait energy image is reported as a good feature which is robust
to silhouette errors and image noise, and is defined by [7]
G(x, y) = 1 N
I(x, y, t) (3)
where N is the number of frames in the sequence I(x, y, t), t is
the frame number, x and y are the image coordinate [7].
Modelling the Effect of View Angle Variation on Appearance-Based
Gait Recognition 811
4.3 KFD Feature Extraction
To extract KFD feature, the outer contour first needs to be
obtained. The outer contour can be easily derived using a
border-following algorithm based on connectivity. Then all the
contours and the gait cycle are normalized to have the same number
(N ) of sample and the same number (T ) of frame, respectively. All
Fourier descriptors g(i) can be obtained by discrete Fourier
transform. The KFDs are defined as in [8]:
G = [ |g(2T )| |g(T )| ,
] (4)
where N is the number of sample points of each contour, and T is
the number of frames in a gait cycle.
5 Experimental Results and Analysis
Each subject has 6 normal walking sequences at each view angle. Wet
put the first 4 sequences into the gallery set, and the last 2
sequences into the probe set. A number of experiments are carried
out to discover the relationship between gait recognition
performance and view angles. Fig. 3 and Table 1 show the CCRs of
the experiments which take GEI as a feature. It can be noticed from
Fig. 3 that there exist two peaks on the CCR curves in each
subfigure. and that CCR reaches the first peak at θp = θg and the
second peak at θp = 180 − θg. A geometrical model is proposed for
explaining the existence of these two peaks. Fig. 4 and Table 2
display the CCRs of the experiments which take KFDs as a feature.
As in Fig. 3 and Table 1, a similar phenomenon can be found in Fig.
4 and Table 2.
We can get that CCRs basically remains a constant CM along the
major diagonal line (θg = θp) of Tables 1 and 2, and a constant Cm
along the major skew diagonal line (θg = 180 − θp) except (θg, θp)
∈ {(108, 72), (90, 90), (72, 108)}.
From Figures 3(f) and 4(f), it can be seen that the CCR basically
remains high when θp varies around 90. Thus, the CCR at the side
view is robust to view angle change with respect to other
views.
Table 1. CCR table (%) for GEI (rank=1)
Gallery Probe angle θp
angle θg 0 18 36 54 72 90 108 126 144 162 180
0 99.2 31.9 9.3 4.0 3.2 3.2 2.0 2.0 4.8 12.9 37.9 18 23.8 99.6 39.9
8.9 4.4 3.6 3.6 5.2 13.7 33.5 10.9 36 4.4 37.9 97.6 29.8 11.7 6.9
8.1 13.3 23.4 13.3 2.0 54 2.4 3.6 29.0 97.2 23.0 16.5 21.4 29.0
21.4 4.8 1.2 72 0.8 4.4 7.3 21.8 97.2 81.5 68.1 21.0 5.6 3.6 1.6 90
0.4 2.4 4.8 17.7 82.3 97.6 82.3 15.3 5.2 3.6 1.2
108 1.6 1.6 2.0 16.9 71.4 87.9 95.6 37.1 6.0 2.0 2.0 126 1.2 2.8
6.0 37.5 33.5 22.2 48.0 96.8 26.6 4.4 2.0 144 3.6 5.2 28.2 18.5 4.4
1.6 3.2 43.1 96.4 5.6 2.8 162 12.1 39.1 15.7 2.4 1.6 0.8 0.8 2.4
5.2 98.4 28.6 180 41.1 19.8 8.1 3.2 2.0 0.8 1.6 3.6 12.5 51.2
99.6
812 S. Yu, D. Tan, and T. Tan
Table 2. CCR table (%) for KFD (rank=1)
Gallery Probe angle θp
angle θg 0 18 36 54 72 90 108 126 144 162 180
0 71.8 5.2 5.2 2.4 0.8 1.2 1.2 0.8 1.6 3.2 33.1 18 3.6 49.2 14.5
4.4 2.8 3.2 4.0 3.6 4.0 8.9 4.0 36 2.8 12.1 72.6 11.7 3.6 2.8 2.0
3.2 14.1 10.9 2.4 54 2.0 3.2 10.5 69.4 7.7 2.4 4.4 14.9 10.9 3.2
0.8 72 0.4 0.8 2.8 12.9 77.8 16.9 25.0 8.9 2.4 1.2 0.0 90 0.4 0.8
3.6 4.4 23.0 75.0 20.6 4.0 2.0 0.8 1.2
108 0.4 2.4 3.2 5.2 20.6 21.4 69.8 10.5 2.8 1.2 0.8 126 0.8 3.6 4.8
14.9 11.7 5.6 14.1 71.4 10.5 3.6 1.6 144 2.0 6.9 16.1 12.1 4.0 2.4
2.8 12.5 71.0 11.7 3.2 162 2.8 10.9 10.9 1.6 2.0 2.4 2.8 6.0 11.3
72.2 3.6 180 30.6 3.2 4.8 1.6 2.0 2.4 1.2 2.8 3.6 7.3 67.7
Gallery View Angle=0
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
Rank=1
Rank=3
Rank=5
Fig. 3. CCRs(%) for GEI(the abscissa is θp, and the ordinate is
CCR)
5.1 A Geometrical Model
Why are there two symmetrical (though not strictly) peaks on the
curves in Fig. 3 and 4? In our opinion, it is the human body
symmetry that results in this phenomenon. Suppose that 3 images
are, respectively, taken from 3 different view angles θ, 180 − θ,
and 180+θ, as illustrated in Fig. 5, and that the cameras are far
away from the subject.
Modelling the Effect of View Angle Variation on Appearance-Based
Gait Recognition 813
Gallery View Angle=0
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
CCR
0 20 40 60 80 100 120 140 160 180 0
10
20
30
40
50
60
70
80
90
100
Rank=1
Rank=3
Rank=5
Fig. 4. CCRs(%) for KFD(the abscissa is θp, and the ordinate is
CCR)
180-
180+
Fig. 5. Images at 3 view angles
S(θ), S(180 − θ) and S(180 + θ) represent the 3 human silhouettes
extracted from the 3 corresponding images. For the distance between
the cameras and the subject is large, we can reasonably
consider:
S(θ) ≈ −→ S (180 + θ) (5)
where the symbol → means flip horizontally. On the other hand,
human body has the symmetry, so we have −→
S (180 + θ) ≈ S(180 − θ) (6)
Walking can partly break this symmetry with respect to a fixed view
angle, which is the reason for using the symbol ≈ in Equation
(6).
814 S. Yu, D. Tan, and T. Tan
From Equations (5) and (6), it is straightforward to get Equation
(7):
S(θ) ≈ S(180 − θ) (7)
This implies that the silhouette from view angle θ is similar to
that from view angle 180 − θ. Thus, when the gallery angle is θ,
using the data from 180 − θ as probe can produce relative higher
CCR, from which a local peak positioned at 180 − θ naturally
occurs, compared with the values of CCR at other view angles
distant from θ.
5.2 A Mathematical Model
Based on the analysis in Section 6.1, there are two peaks on CCR
curves in Figures 3 and 4. By observing the shapes of curves, we
can reasonably use a mixed Gaussian function to model CCR
curves.
The analytical expression of the mathematical model is defined
as:
f(θg, θp) = CMe− (θg−θp)2
2σ2 + Cme− (180−θg−θp)2
2σ2
2σ2
] (8)
where CM and Cm are the same as previous definitions. and σ is
treated as a constant in our experiments, which indicates the level
of performance deterioration when the probe view departs from the
gallery view. The value of σ is optimized by the Curve
Fitting
Toolbox in Matlab, and it takes 15 here. CM e− (θg−θp)2
2σ2 and Cme− (180−θg−θp)2
2σ2 are
two Gaussian functions which simulate the two ridges in Figures 6
and 8. 1−e− (θg−θp)2
2σ2
is a weighting term which makes sure that f(θg, θp) does not exceed
unity. The CCRs in Table 1 are shown in Fig. 6. Fig. 8 displays the
CCRs in Table 2.
Fig. 7 and Fig. 9 are the continuous versions of Fig. 6 and Fig. 8
obtained from our mathematical model (equation (8)), respectively.
The theoretical results computed from Equation (8) generally
conform to the experimental ones in Table 1 and Table 2.
0 18 36 54 72 90 108 126 144 162 180
0 18
36 54
72 90
108 126
144 162
180 0
)
Fig. 6. CCRs for GEI Fig. 7. The modelled CCRs for GEI
From Equation (8), Fig. 7 and Fig. 9, there are two perpendicular
ridges which super- pose each other. It is this superposition that
makes the CCR around the site (90, 90) much higher than in other
regions. Thus, the CCR at the side view is robust to view angle
variation.
Modelling the Effect of View Angle Variation on Appearance-Based
Gait Recognition 815
0 18 36 54 72 90 108 126 144 162 180
0 18
36 54
72 90
108 126
144 162
180 0
)
Fig. 8. CCRs for KFD Fig. 9. The modelled CCRs for KFD
6 Conclusions and Future Work
In this paper, we have presented an analysis of the effect of view
angle variation on the performance of appearance-based gait
recognition methods the proposed and mod- els. The novelty of our
work is three-fold: first, it is a systematic study on multi-view
gait recognition; secondly, it investigates how the performance is
affected by view an- gle changes with a useful mathematical model
depicting the relationship between view angle and
performance(despite the current simplicity); last but obviously not
least, it an- swers two open questions: why the side view is more
suitable to recognize human gaits, and how view angle variation
impacts the gait recognition performance. Our future work will be
focused on view-invariant gait feature extraction, a better
mathematical model taking into account the effect of θg, θp and
features on σ, and the establishment of a multi-view gait database
in an outdoor environment.
Acknowledgement
This work is partly supported by National Natural Science
Foundation of China (Grant No. 60335010), National Basic Research
Program of China (Grant No. 2004CB318100) and International
Cooperation Program of Ministry of Science and Technology of China
(Grant No. 2004DFA06900).
References
1. Murray, M.P.: Gait as a total pattern of movement. American
Journal of Physical Medicine 46 (1967) 290–332
2. Sarkar, S., Phillips, P.J., Liu, Z., Vega, I.R., Grother, P.,
Bowyer, K.W.: The humanid gait chal- lenge problem: Data sets,
performance and analysis. IEEE Transactions on Pattern Analysis and
Machine Intelligence 27 (2005) 162–177
3. Johnson, A.Y., Bobick, A.F.: A multi-view method for gait
recognition using static body parameters. In: Proc. of 3rd
International Conference on Audio and Video Based Biometric Person
Authentication, Halmstad, Sweden (2001) 301–311
816 S. Yu, D. Tan, and T. Tan
4. Kale, A., Chowdhury, A.K.R., Chellappa, R.: Towards a view
invariant gait recognition algo- rithm. In: Proc. of IEEE
Conference on Advanced Video and Signal Based Surveillance, FL, USA
(2003) 143–150
5. Shutler, J.D., Grant, M.G., Nixon, M.S., Carter, J.N.: On a
large sequence-based human gait database. In: Proc. of the 4th
International Conference on Recent Advances in Soft Computing,
Nottingham, UK (2002) 66–72
6. : (Center for biometrics and security research, casia.
http://www.cbsr.ia.ac.cn) 7. Han, J., Bhanu, B.: Statistical
feature fusion for gait-based human recognition. In: Proc. of
the 2004 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition, Washington DC, USA (2004) II842–II847
8. Yu, S., Wang, L., Hu, W., Tan, T.: Gait analysis for human
identification in frequency domain. In: Proc. of the 3rd
International Conference on Image and Graphics, Hong Kong, China
(2004) 282–285
9. Wang, L., Tan, T., Ning, H., Hu, W.: Silhouette analysis-based
gait recognition for human identification. IEEE Transactions on
Pattern Analysis and Machine Intelligence 25 (2003) 1505–1518
Introduction
LOAD MORE