Towards Accurately Extracting Facial Expression Parameters Xuecheng Liu Dinghuang Ji Zhaoqi Wang and Shihong Xia {liuxuecheng,jidinghuang,zqwang,xsh}@ict.ac.cn Institute of Computing Technology, Chinese Academy of Sciences, China
Jan 03, 2016
Towards Accurately Extracting Facial Expression Parameters
Xuecheng Liu Dinghuang Ji Zhaoqi Wang and Shihong Xia{liuxuecheng,jidinghuang,zqwang,xsh}@ict.ac.cn
Institute of Computing Technology, Chinese Academy of Sciences, China
Outline
Background Problem Our Method Experiments Summary
Background
Applications of facial animation
Movies Games Tele-Immersion
Background
• Blendshapes
1
0.8w
2
0.5w
...
Background
Vicon Captured Data
Head Motion
Rotation Transition
Facial Expression
Blendshapes Weights
ProblemUsually,
• Head absolute orientation is first extracted, including head rotation matrix R and transition vector t.
• The captured facial expression data are then transformed to local face coordinates system.
• However, the assumption that four markers are relative fixed may not always be true. This results to error of computing head orientation. Then, above error will be amplified during computing expressions parameters w.
Problem
• [HAV 06] propose a hierarchical manner– Doing a global or gross stabilization choosing
selective markers (head, ears and the nose bone)– Refining the output by local or fine stabilization by
analyzing the marker movements relative to a facial surface model
Our Method
• To undermine the error accumulation, we propose a revision based optimization method.
Our Method
• To undermine the error agglomeration, we propose a revision based optimization method.Simplify
Our Method
• To undermine the error agglomeration, we propose a revision based optimization method.
• Then we use a traditional alternative method and non-negative least square to solve the problem.
Our Method
• Linear blendshape
• Nonlinear blendshape
Experiments
• Data and Environment– Training data: 500 frames– Testing data: 15 segments( #Frame
>36,000)– Ratio >= 80– Basic blendshapes: 27个– Hardware : x86 2.67G Core2 CPU, 2G RAM– Software:Matlab
Experiments
• We compare with two published which are both representative works in areas.– [HAV 06] is a typical linear blendshape based
method, this work is presented as course of Siggraph 2006 and used in many popular movies.
– [LXFW 11] is a typical nonlinear blendshape based method, and is published in CGF 2011.
Experiments
• Contrast experiment 1 (with [HAVALDAR 06])[HAVALDAR 06] [Our Method]
Experiments
• Contrast experiment 1 (with [HAV 06])
0%2%4%6%8%
10%12%14%16%18%20%22%
Faci al ani mat i on cl i p
Rela
tive
Err
or
[Hav06] Our Method
0. 00. 20. 40. 60. 81. 01. 21. 4
Faci al ani mat i on cl i p
Abso
lute
Err
or(m
m)
[Hav06] Our Method
Absolute Error
Relative Error
(b)(a)
Experiments
• Contrast experiment 1 (with [HAV 06])
0
5
10
15
Faci al ani mat i on cl i p
Aver
age
Iter
atio
ns
[Hav06] Our Method
020
4060
80100
120
Faci al ani mat i on cl i p
Fram
e ra
tes
f/s
()
[Hav06] Our Method
(b)(a)
Experiments
Experiments
• Contrast experiment 2 (with [LXFW11])[LXFW11] Our Method]
Experiments
• Contrast experiment 2 (with [LXFW11])Absolute Error
Relative Error
(b)(a)
0. 0
0. 5
1. 0
1. 5
2. 0
2. 5
Faci al ani mat i on cl i p
Abso
lute
err
ormm
()
[LXFW11] Our Method
0%
5%
10%
15%
20%
25%
30%
35%
Faci al ani mat i on cl i p
Rela
tive
err
or
[LXFW11] Our Method
Experiments
• Contrast experiment 2 (with [LXFW11])
(b)(a)
0
5
10
15
20
Faci al ani mat i on cl i p
Aver
age
iter
atio
ns
[LXFW11] Our Method
0. 0
10. 0
20. 0
30. 0
40. 0
50. 0
Faci al ani mat i on cl i pFr
ame
rate
sf/
s(
)
[LXFW11] Our Method
Experiments
Conclusion
Application Methods Precision promotion
Efficiency promotion
Linear blendshape
Our Method VS. [Hav06]
+12.3% +86.0%
Nonlinear blendshape
Our Method VS. [LXFW11]
+23.5% -18.2%
Bibliography
• [HAV 06] HAVALDAR P. Sony pictures imageworks. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Courses (New York, NY, USA, 2006), ACM, p. 5.
• [LXFW11] Xuecheng Liu, Shihong Xia, Shihong Xia, Yiwen Fan, Zhaoqi Wang. Exploring nonlinear relationship of blendshape facial animation. To appear in the Journal of Computer Graphics Forum.