-
Pose Synthesis Using the Inverse of Jacobian Matrix
Learned from Examples
Chunpeng Li∗
Institute of Computing Technology,Chinese Academy of
Sciences
Graduate School,Chinese Academy of Sciences
Shihong Xia†
Institute of Computing Technology,Chinese Academy of
Sciences
Zhaoqi Wang‡
Institute of Computing Technology,Chinese Academy of
Sciences
ABSTRACT
This paper presents a method of pose synthesis based on a
low-dimensional space and a set of characteristics of motion
learnedfrom examples. This method consists of two phases: learning
andsynthesis. In the learning phase, a low-dimensional and
discreterepresentation of the space of natural poses is constructed
by us-ing a Self Organizing Map (SOM). Meanwhile, a set of
matricesis extracted from the motion data. These matrices describe
howthe poses change with the end-effectors’ positions, and play a
keyrole in synthesizing natural looking results. In the synthesis
phase,a lightweight algorithm based on the learned parameters is
used.The synthesis process is very efficient because there is no
time-consuming calculation, like numeric optimization or matrix
invert-ing. Compared with other methods, our method not only can
pro-duce natural looking poses in real-time, but also works well
withconstraints positioned in a larger range. We apply our method
inapplications of interactive pose editing, real-time motion
modifica-tion, and pose reconstruction from image. The results have
proventhe robustness and effectiveness of our method.
Keywords: Inverse Kinematics, Self Organizing Map,
JacobianMatrix, Character Animation.
Index Terms: I.3.7 [Computer Graphics]:
Three-DimensionalGraphics and Realism—Animations; I.2.6 [Artificial
Intelligence]:Learning—Connectionism and neural nets
1 INTRODUCTION
In the past decade virtual reality applications have emerged in
manyareas, such as entertainment, training, communication and so
on.In these applications, virtual human, as the digital
representation ofhuman, is one of the most important elements to
improve the inter-active capability and immersive experience.
However, it remains tobe a challenging issue faced by modeling
virtual human to synthe-size natural motions including interactive
motions with virtual en-vironment and with the human user. In this
paper, a new approachto synthesizing and controlling virtual
character poses from a set ofconstraints is presented.
In virtual environment, a virtual character is often modeled as
aset of rigid segments connected by joints. Given a set of joint
an-gles, we can calculate the posture of the character, which is
calledforward kinematics. On the contrary, the problem of computing
aset of joint angles to position the end-effector at a desired
locationis called inverse kinematics. Inverse kinematics (IK) is
more intu-itive and simpler because it operates the positions of
few segmentsin 3D space rather than the angles of each joint in
angular space.
∗e-mail: [email protected]†e-mail: [email protected]‡e-mail:
[email protected]
Figure 1: A simple demonstration of our method. One end of
themodel is fixed at the original point. Top row: One example
motionis given, shown in the left column. The poses in the right
columnare the outputs by our method. Bottom row: A different
example isgiven. Our method generates a set of new poses.
The solver of inverse kinematics problem is one of the most
im-portant tools for interactive control of characters in virtual
realityapplications, such as interactive games, workspace
computing, andvirtual space teleconferences.
Given a vector p of known joint variables, also named
config-uration or pose of the model, the forward kinematics problem
ofcomputing the position vector g of the end-effector is a simple
mat-ter of matrix concatenation and the process is denoted as:
f (p) = g (1)
But the IK problem is to place the end-effector at a
specifiedposition g, named geometrical constraint. Determining the
appro-priate joint variable vector p to achieve the goal requires a
solutionto the inverse of equation (1):
p = f−1(g) (2)
Solving the IK problem is not easy because that the functionf
(·) is nonlinear and there may be many p’s for a particular g for
aredundant structure. This paper focuses on how to search for a
posefor a virtual character satisfying constraints in the space of
naturalposes.
Inverse kinematics originated from robotics field. The
manipu-lation of robot arms with 6 up to 9 degrees of freedom (DOF)
waswell solved. However, skeletons of virtual human are much
morecomplex which usually have more than 40 DOFs. Although
manymethods have been proposed to solve inverse kinematics
problemsin the field of character animation, little research have
been downto try to extract characteristics out of real motion data
for inversekinematics control.
99
IEEE Virtual Reality Conference 2007March 10 - 14, Charlotte,
North Carolina, USA1-4244-0906-3/07/$20.00 ©2007 IEEE
-
We propose a new method to solve the inverse kinematics prob-lem
by extracting characteristics from real motion data. The mainidea
of our approach is to learn the inverse of Jacobian matrix
overposes. The Jacobian matrix of a pose indicates how the
end-effectorposition varies with respect to each joint angle near
the pose, whichis calculated numerically in many traditional
solvers. A simpledemonstration is shown in Figure 1.
Our approach consists of two phases: learning and synthesis.In
the learning phase, we project each pose of examples into
two-dimensional nodes using Self Organizing Map (SOM). The
weightsof each node represent a cluster of human poses. At the same
time,a local Jacobian matrix inverse is learned for each node. In
the syn-thesis phase, given a desired position of end-effector, a
referencepose is obtained by searching these nodes whose
end-effector po-sition is nearest to the desired position. Then, an
iterative processusing the learned matrices is executed until the
goal is reached. Thefeatures of our method are outlined as
follows.
• Automation. This method can learn from examples automat-ically
without guidance. And there is no need to preprocessthe examples,
such as choosing an active set [3] or creatingparameterized motions
[7].
• Natural results. The poses produced by this method are
nat-ural, because the characteristics extracted from real
motionexamples are used in the synthesis process.
• High efficiency. This method can generate required poses
inreal-time, because it does not contain time-consuming pro-cesses
such as nonlinear optimization or matrix inverting.
• Robustness. Previous methods based on blending or
inter-polation usually can not create natural poses when the
con-straints are much different from examples. However, thismethod
works effectively with a larger range of positioningconstraints
compared with these methods.
• Repeatability. If a given constraint always generates thesame
pose, the system is said to be repeatable. This methodis repeatable
because of having an initializing step in the syn-thesis phase.
2 RELATED WORK
The inverse kinematics for controlling robot manipulators has
beenstudied extensively in robotics field and many analytical
methodshave been proposed [2]. These methods are very efficient,
but theycan not deal with the models with complicated
structures.
Tolani [13] used a combination of analytical and
numericalmethods to solve the IK problem for anthropomorphic limbs.
Thismethod can solve problem in real-time and allow the user to
exploreall possible solutions. But the main limitation of this
method is thatit is only suitable for models with up to 7 DOFs.
Iterative methods are more general, which can deal with
anymodel. These methods can be divided into three subclasses
further,namely Cyclic Coordinate Descent (CCD) method, Jacobian
matrixbased method and optimization based method. CCD method [6] is
aheuristic iterative method, which optimizes only one joint in
everysingle step to minimize the total objective function. This
methodis relatively fast in computation and immune to
singularities. How-ever, because of the heuristic strategy adopted
by CCD, rotationsare not equally distributed throughout the
kinematical tree whichwill make the result unnatural. Jacobian
matrix based method dif-ferentiates the nonlinear objective
function by constructing Jaco-bian matrix and inverting it.
Differential equations are then solvedby numerical integration
[17]. In order to overcome the shortcom-ing of being highly
time-consuming and the singularities caused byinverting Jacobian
matrix, Jacobian transpose based method [19]
and Damped Least Square method [15] were proposed. The for-mer
uses Jacobian matrix transpose instead of Jacobian matrix in-verse
and the latter adds a damping term to avoid singularities.Yamane
[20] proposed a method which can add and remove con-straints at
will. Nonlinear optimization techniques are also used tosolve
inverse kinematics problems. Zhao’s work [21] is one of themost
general methods. They define the distance between the
currentposition of the end-effector and the goal as the objective
function,and the joint angles as the variables. Then, modified BFGS
is usedto solve the minimization problem. This approach is
applicable toany tree structured system and can be added with other
kinds ofconstraints easily. But it will slow down rapidly as the
number ofvariables increases.
In recent years, with the prevalence of the motion capture
de-vices, techniques based on examples have been developed.
Thesemethods take a step in improving the naturalness of the
results.
Interpolation is one of the most effective ways to use
examples.Rose et al. proposed a system blending example motions by
com-puting a best fit map between blending weights and motion
parame-ters [10]. Kovar’s approach [7] can synthesis new motion
sequencesby interpolating examples based on automated analysis and
param-eterization of motions in large data sets. These methods have
theadvantage of high efficiency, but lack an accurate inverse
mappingfrom constraints to blending weights. Therefore, they have
to gen-erate a dense sampling of parameter space to get an
approximateresult. Recently, a novel motion interpolation technique
[14] hasbeen proposed based on geostatistics, which treats motion
inter-polations as statistical predictions of missing data in an
arbitrarilydefinable parametric space. This method can produce
accurate mo-tions with given control parameters as well as give an
estimationof reliability. However, the accuracy of result relies on
sufficientnumber of example motions. It requires many samples
distributedaround the center of the control space to get a good
result, which isalso a drawback of interpolation based methods.
Traditional methods are extended with the utilization of
motioncapture data. The key idea of Komura’s work [6] is to
extracta weight matrix from real motion data based on Jacobian
matrixmethod. The weight matrix determines the motion of the
redun-dant joints and the transformation functions that define the
motionof the end effectors. Though this idea is similar to our
method inextracting the characteristics of the motion from
examples, the au-thors do not analyze the space of the natural pose
and do not con-nect it with weighting matrix, which limits the
range to positionconstraints. Example guided inverse kinematics
(EGIK) systemproposed in [12] is an extension of Zhao’s work [21].
They takeinto account the distances to examples when constructing
the ob-jective function. This method can solve IK problem with
humanmodels having different segment lengths, but it is difficult
to satisfythe requirement of real-time application because of using
nonlinearprogramming techniques.
Approaches based on a learned low-dimensional space are pop-ular
recently. Style-based inverse kinematics [3] describes a
novelmethod which synthesizes new poses in a low-dimensional
prob-abilistic space. Given training poses, a scaled Gaussian
processlatent variable model (SGPLVM) is used to represent the
proba-bility distribution function over poses. In the synthesis
phase, anobjective function derived from the model is optimized to
searchfor a new pose which not only satisfies constraints but also
hasthe maximum likelihood. Another work, called mesh-based
inversekinematics [11], was proposed to manipulate meshes sharing
thesame idea. Their approach can find meaningful mesh
deformationsthat meet specified vertex constraints efficiently. The
algorithm firstconstructs a space of natural deformations using
feature vectors ex-tracted from example meshes, and then an
optimization techniqueis employed to search for a result in the
space as close as possible tothe training examples. Both of the
methods have a similarity to ours
100
-
Figure 2: A demonstration of our method. Left: The original
motion sequence. Each yellow marker represents the position of the
right hand ineach frame. Middle: Learning results. Each yellow
marker represents a reference pose learned by SOM. Right: A set of
poses synthesized inreal-time by dragging the right hand with
mouse. The positions of the right hand form two letters ”VR” which
lie on a plane.
in solving the IK problem within low-dimensional spaces which
de-scribe the space of natural poses or appropriate deformations.
Butthey are different from ours in the ways of modeling and
synthesiz-ing. Our method uses a discrete space and a lightweight
iterativeprocedure instead of continuous spaces and nonlinear
optimizationprocedures for modeling and synthesizing separately,
which cangain an improvement in efficiency.
Many algorithms based on machine learning for control of an
in-dustrial robot have been developed in robotic field, among which
isRitter’s work for visuo-motor control [8]. The objective is to
teachthe robot arm to position its end-effector using information
gainedsolely from the pair of cameras. Neither an external teacher
norany prior information about the geometry of the robot arm or
thecameras was employed. Our work is based on this work and
ex-tended from it. Firstly, structure for human model is much
morecomplicated than that of robot arm and multiple constraints
prob-lem is considered. Secondly, the meaning of the SOM weights
isdifferent. Thirdly, we extend the SOM learning algorithm by
usingquaternion which is more suitable for joint angles.
3 OVERVIEW
As mentioned above, motion rate control is one of the most
com-mon methods solving the inverse kinematics problem. This
methodlinearizes the problem about the current configuration, which
canbe denoted by the following equation:
∆g = J(p) ·∆p (3)
where
J =∂ f
∂ p(4)
Inverting the relationship of equation (3) provides the basis
forresolved motion rate control.
∆p = J−1(p) ·∆g (5)
If the inverse of J is known, we can compute incrementalchanges
in the joint variables that produce a desired incrementalchange in
the end-effector position. Note that equation (5) standsonly when
the Jacobian is both square and non-singular. Whena structure is
redundant or when its pose passes through a singu-lar
configuration, the Jacobian matrix cannot be inverted. In
bothcases, some generalized inverses of Jacobian J+ are used to
finda useful solution [15]. In the following of this paper, the
matrixlearned by our method is also a kind of generalized inverse
of Jaco-bian, but we drop the distinction between them for
notational sim-plicity.
A simple iterative scheme for solving the inverse
kinematicsproblem can be based on equation (5). In each step, a
desired ∆g canbe computed from the current and desired end-effector
positions.The joint incremental change ∆p can then be computed
using theinverse Jacobian and integrated once to find a new joint
state vectorp. The procedure is repeated until the end-effector has
reached thedesired goal. Note that since the linear relationship
represented byJ is only valid for small perturbations in the
manipulator configura-tion, J−1(p) must be recomputed in each step
[16].
The main idea of our approach is to learn the inverse of
Jaco-bian matrix J−1(p) over poses from motion capture data instead
ofcomputing numerically. Synthesizing new poses using the
learnedmatrices can get more natural results and avoid computing
the Jaco-bian matrix and its inverse. Our approach consists of the
followingtwo phases.
Learning Examples. Because of the complexity of inputspace, it
is difficult obtain the continuous Jacobian matrix inverseover the
entire space. However, SOM is an effective technique toanalyze the
complex space, which can reduce the dimension of thespace as well
as discrete it. The output of the SOM defines a low-dimensional
representation of the input data. The weight vector ofeach neural
unit, called reference pose, can be treated as a clustercenter of
input data. Besides the weight, each neural unit has a ma-trix used
to store the inverse of Jacobian matrix nearby the referencepose.
The details of the learning process are described in the
nextsection.
Pose Synthesis. In order to generate new poses that reach
thegoal, we first search for a pose in training results as an
initial con-figuration. Then with the local Jacobian matrix
associated with theselected pose, the residual deviation from the
desired target loca-tion is transformed into a correctional fine
movement. Repeat thefine movement iteratively until the target is
reached or no furtherreduction of the distance could be made.
Figure 2 demonstrates theprocess of our approach.
4 LEARNING PHASE
4.1 Learning the Reference Poses
SOM is a data visualization technique proposed by Teuvo Ko-honen
[5] which can reduce the dimensions of data using self-organizing
neural networks. SOM can transform the input of ar-bitrary
dimension into a low-dimensional discrete map subject to
aneighborhood preserving constraint.
A SOM consists of two layers of neurons: an input layer andan
output layer. Each neuron of the input layers is connected with
101
-
each neuron of the output layer using a weight. The weights
con-nected with a output neuron are interpreted as a reference
vectorin the input space. The output layer is a one-dimensional or
two-dimensional discrete grid. The learning process is competitive
andunsupervised. When an input pattern is presented to the
network,one neuron in the output layer is determined whose
reference vectoris the closest to the input pattern. This neuron is
called the winnerneuron and the weights of itself and neurons
around it are movedcloser to the input pattern. Readers can refer
to [4] for more details.We summarize the features of SOM here
briefly.
• From the dimension reduction point of view, SOM is an
ef-fective nonlinear dimension reduction method. Data from
ahigh-dimensional input space is mapped onto a usually
one-dimensional or two-dimensional output space, trying to findthe
principal curves or principal surface of the input data [9].
• At the same time, the projection from the continuous
inputspace to the discrete output space leads to an automatic
clus-tering of the data, i.e. similar data samples are stored in
prox-imity thereby forming clusters. This feature makes the
learn-ing of Jacobian matrix inverse easy.
• The output of SOM can preserve the topological order of
theinput samples. Topological order means that similar inputsamples
are close to each other while dissimilar inputs arefar from each
other.
The standard learning algorithm uses the Euclidian distance
tomeasure the difference of input vectors and to modify the
weights.However, we usually use quaternions to represent joint
rotations,which can not be measured by Euclidian distances. Refer
to [1] forthe basic definitions and operations on quaternion. For
this reason,the learning algorithm is modified as following.
Suppose that the map consists of a set of N neuron units
la-beled by indices i (i = 1,2, ...N). The weights of neuron i are
de-noted as wi = {wi1,wi2, ...wiM}, where wim is the rotation of
jointm (m = 1,2, ...M). These weights are randomly initialized in
unitquaternion space at the beginning. Each learning step mainly
con-tains 4 steps:
1. Given a pose x = {x1,x2, ...xM} of the original data as
theinput of the map, compute the distance between the pose andthe
reference pose of each neuron unit. The distance functionis defined
as:
dist(x,wi) =
√
√
√
√
M
∑m=1
‖λm · log(x∗m ·wim)‖2 (6)
where λm is a weight indicating the importance of the joint mand
x∗ is the conjugate of x.
2. Find the winner or excited neuron unit (competitive
process):
c = argmini{dist(x,wi)} (7)
3. Determine the neighbor function (cooperative process):
Λc,i(n) =
{
1−dc,i
R(n)dc,i ≤ R(n)
0 dc,i > R(n)(8)
where dc,i is the distance between the unit c and the unit i
inthe output layer, and R(n) is the radius of the updating
range.
4. Update the weights:
wim(n+1) = Slerp(wim(n),x(n),η(n) ·Λc,i(n)) (9)
where Slerp(p,q, t) = (q · p∗)t · p and η is the learning
rate.In the learning process, the learning rate η(n) and the
radiusR(n) decrease with the increasing of the iteration step
n.
4.2 Learning the Jacobian Matrix Inverse
We assume that the poses belong to the same neuron share the
sameJacobian matrix inverse. The assumption is based on the fact
thatthese poses are similar and the Jacobian matrix is a continuous
func-tion over a space of poses. This assumption can be represented
asthe minimization of the following error function:
E =1
2e2 =
1
2∑(∆p− J+ ·∆g)2 (10)
where ∆p means the difference between the reference pose of
aneuron and a pose belongs to it, and ∆g denotes the displacementof
the end-effector’s position between them. The form of ∆p couldbe a
set of Euler angles or quaternions, and we use the former onein the
following paragraphs for simplicity.
In order to learn the Jacobian matrix in the process of
construc-tion of the SOM, the extended SOM is employed. Each neuron
ofthe output layer is associated with a matrix structure which is
usedto store J+. The update rule of J+ is given by an error
correctionrule of Widrow-Hoff type [18], which is based on the use
of instan-taneous values for the cost function. At the nth step,
the inverseof Jacobian matrix is denoted as J+(n). The object of
learning istrying to minimize the following equation instead of
equation (10):
E(n) =1
2∑(∆p− J+(n) ·∆g)2 (11)
The updating rule is given by:
J+i (n+1) = J+i (n)+η(n) ·Λc,i(n) ·∆J(n) (12)
where∆J(n) = (∆p− J+i (n) ·∆g)∆g
T ‖∆g‖−2 (13)
is the partial derivative of E(n) at J+(n).As explained in
equation (5), methods directly based on Jacobian
matrix inverse usually suffer difficulties when the model is
redun-dant or its configuration is close to singularities. Our
method canavoid the first problem and generate stable results as
long as theexamples near the singularities are given. Because we
can use thelearned matrix to calculate the increment of each joint
angle cor-responding to the constraints directly, there is no need
to calculatethe inverse or pseudoinverse of Jacobian matrix.
Moreover, becauseof the ”cooperative” property in the learning
process of SOM, therelationship between the joints and
end-effector’s position near thesingular configurations can also be
extracted from the examples.That is, the stability of our method is
based on a priori knowledgeand on the definition of preferred
direction of motion near singular-ities. However, our method does
not work in some extreme situa-tions where all the poses in given
training motions have the sameend-effectors’ positions. That means
‖∆g‖ equals to zero for anytwo poses. In this situation, equation
(10) is degenerated, so theredoes not exist a proper J+. But in
practice, these situations rarelyoccur.
5 POSE SYNTHESIS
5.1 Synthesis
Two kinds of parameters are obtained in the learning phase, a
set ofreference poses wi and a set of Jacobian matrices inverses
J
+i asso-
ciated with each pose. wi are the low-dimensional
representationsof the space of natural poses, which are used to
initialize the con-figuration. J+i describe how the joints vary
with the end-effector’sposition near the pose wi, which are used to
refine the configura-tion. Using these parameters in the synthesis
phase not only accel-erates the solving process, but also improves
the naturalness of theresults. The synthesis of a new pose given
the desired position ofthe end-effector consists of two parts
according to the parametersused, namely an initializing step and a
refinement process.
102
-
Initialization. This step is used to initialize the
configurationby searching for a pose wc, whose end-effector’s
position is theclosest to the desired position. We denote the
desired position ofthe end-effector as utgt . This step can be
presented as:
p0 = wc (14)
where
c = argmini‖utgt − f (wi)‖ (15)
The subscript of p0 indicates the number of iteration.
Refinement. This process is iterative. In each step, it takesa
small step towards the goal. The position of the end-effector uican
be computed by forward kinematics ui = f (pi). The distancebetween
it and the goal is ∆u = utgt − ui. The refinement processcan be
represented as the following equation.
pi = pi−1 + J+c ·∆ui−1 (16)
Because the learned matrices are local, which are not very
reli-able for large values of ∆u, we will take smaller steps by
adding aparameter α(0 ≤ α ≤ 1) as following.
pi = pi−1 +α · J+c ·∆ui−1 (17)
We can use a linear search algorithm to determine the value ofα
in each step. Repeat this procedure until ∆ui is small enough
orthere is no further steps could be taken. In the latter
situation, theobjective function reaches a local minimal which
usually occurs insolving multi-constraints problems.
However, it will generate discontinuous results even when
weposition the constraint continuously because of the initializing
stepin each synthesis process. This problem can be overcome
byweighting poses around wc.
p0 = ∑i
hc(i) ·wi (18)
pi = pi−1 +α · (∑i
hc(i) · J+i ) ·∆ui−1 (19)
where hc(i) is the weight function which has a maximum value
atpoint c and decrease to zero as the distance between c and i
in-creases.
5.2 Multiple constraints
In many applications, we need to set multiple constraints on the
po-sitions of different end-effectors of a model. For example,
duringpose editing, it would be useful to be able to specify some
partsof the body fixed. Many situations can be viewed as
synthesisproblems with multiple constraints. Though our method is
not re-stricted to solo-constraint situations, it usually can not
converge toa right answer if we use it to solve multi-constraints
problems di-rectly. That is, the iteration process stops while the
end-effectors donot reach the goals. These situations are caused
due to the trendscaused by different goals are conflict. For
example, if the motionlearned by SOM only with one hand moving
rightward while onefoot moving upward, then confliction occurs when
we are trying tomove the hand rightward and move the foot downward.
The incre-mental change by trying to move the hand rightward causes
the footmoved upward and vice versa. When the two forces are equal,
theobjective function can not be reduced.
Here we propose a method to overcome this problem which isbased
on the observation of the dependencey of the joint
angles.Obviously, there is less dependency between joint angles on
dif-ferent kinematic chains. We can fix one arm while move one
legarbitrarily in a large range. Secondly, on the same kinematic
chain,
Figure 3: Poses synthesized by our method. The constraints of
thefirst column are near to the body, while the constraints of the
secondcolumn are far from the body. Each figure contains the front
view andthe side view.
the joint closer to the end-effector changes more easily. For
multi-ple constraints problems, we introduce a weight matrix to
indicatethe relationships. The effect of using weight matrix is the
sameas the traditional method essentially. The elements of the
Jacobianmatrix are zeros or small when the joints are not on the
chain of thecorresponding constraints. In this way, the multiple
constraints canbe satisfied simultaneously.
Given a set of geometry constraints, the target position of
con-
straint k is denoted as uktgt . In the learning phase, we learn
the in-verses of Jacobian matrices of each constraint separately,
denoted
as Jk+. The weight matrix of constraint k is Wk. Then the
synthesisprocess can be represented as:
pi = pi−1 +α ·K
∑k=1
Wk · Jk+c · (u
ktgt −∆u
ki−1) (20)
6 EXPERIMENTS
We have implemented a prototype pose synthesis system in C++
ontop of Windows XP and OpenGL.
Figure 1 demonstrates the principle of our method. Two exam-ples
generated by computer are used to train a linkage model with 4bone
segments and 12 DOFs. Two sets of poses with different con-straints
are generated based on the examples separately. Note thatthe
end-effector moves along the green axis in a small range in
bothexamples, but our method is able to generate new
configurationseven when the constraints are positioned under the
red axis. Theseexperiments also demonstrate the stability of our
method near sin-
103
-
Figure 4: Reference poses outputted by SOM. The poses on the
corners are zoomed in from another viewpoint.
Figure 5: Comparison of outputs synthesized by our method and
DLSmethod. Characters colored cyan (left column) show the
synthesizedposes of our method while blue characters (right column)
representthe output with DLS method. The positions of constraints
in each roware the same.
gular configuration. The first pose of output column in each row
isa result when we drag the end-effector moving near a singular
con-figuration. Different examples cause different actions of the
modelnear the same singularity. However, if we put the two
examplestogether to SOM, our method can not produce a correct
result be-cause that the assumption mentioned in section 4.2 is not
satisfiedany more.
Other experiments are performed on a human model with 19bone
segments and 60 DOFs. Figure 4 demonstrates the referenceposes
learned from a bend-over motion using SOM without anypreprocessing.
This example clip contains 2,235 frames sampledat 120Hz, which is
obtained from CMU motion capture database.The output of the SOM is
a 10×10 array. The learning rate changesfrom 1.0 to 0.02 in the
learning phase. We train the map with 30,000steps.
The training output (Figure 4) shows the low-dimensional
struc-
Figure 6: Comparison of convergence rate between our method
andDLS method.
ture of the motion. We can find out that the position of the
righthand varies from high to low along the X axis and the
orientation ofthe body changes from right to left along the Y axis.
These trendscan be viewed more clearly in the middle of Figure 1.
Obviously,after the learning phase, the unorganized motion data
became orga-nized.
Some synthesized poses with different constraints based on
thebend-over motion are shown in Figure 3. Although some
con-straints are far away from that of the original data, our
method canstill produce plausible results. This demonstration shows
that thelearned matrices can describe the characteristic of the
original mo-tion effectively. Figure 5 shows some snapshots
comparing the out-puts of our method with that of DLS method. The
results demon-strate the superiority of our method in naturalness
of results.
In order to demonstrate the efficiency of our method, we
com-
104
-
Figure 7: A kick motion is modified based on the trajectory of
the right foot. Left: The original motion and the trajectory of the
right foot showedin yellow line. Right: The result synthesized by
our method based on the modified trajectory showed in green
line.
pare it with the DLS method. Three groups of trials with
differentconstraints are executed using both methods. Figure 6
shows theposition error versus steps of iterations. From this
figure, one cansee that our method requires more iterations than
DLS. The maxiteration steps of our method is 90 while that of DLS
method isonly 26 when errors reach to 10−5. However, our method is
fasterthan DLS method because of avoiding the calculation of matrix
in-verse. We execute these trials 1000 times and the average time
foreach execution is presented in Table 1. These experiments are
per-formed on a PC with Core Duo T2300E CPU and the memory of1G.
The time spent of DLS method is about 20 times than that ofour
method.
Table 1: Comparison of time spent.(Unit:ms)
Test 1 Test 2 Test 3 Average
DLS Method 5.313 5.609 5.188 5.370
Our Method 0.296 0.235 0.250 0.260
7 APPLICATIONS
7.1 Interactive Pose Editing
Interactive pose editing is one of the basic applications of our
sys-tem. The experiments above show the application of
positioningthe right hand of a bend-over motion. We have
implemented it witha friendly interface, where we can use mouse to
drag the right handdirectly (Figure 2).
7.2 Real-time Motion Modification
Because of the high efficiency of our method, it can be used to
mod-ify motion data in real-time. Figure 7 is a demonstration of
modi-fying a kick motion. A new kick motion is generated according
tothe modified trajectory of the right foot. Note that there is
only oneexample motion fed to SOM without any preprocessing. Having
nostrict requirements on the quantity and quality of the examples
is anadvantage of our method, but it will also have some influence
on theresults. In this experiment, our method can not generate
plausibleresult when the new trajectory is far away from that of
the trainingexample. The reason for the problem is that the
Jacobian is a localparameter, which is only valid for limited range
of some configura-tion. Given more examples will improve the
results and enlarge therange to position constraints.
Additionally, combining our method with techniques such as
tra-jectory displacement and motion interpolation can produce
moresmooth motion sequences, which is very useful to make
various
interaction motions of virtual character according to the
virtual en-vironment.
7.3 Pose Reconstruction from Image
Our method can also be used to reconstruct the pose from a 2D
im-age of a person. Given a photograph associated with its
projectionmatrix, we can reconstruct the pose through a few
positions of end-effectors specified by the user interactively. In
order to realize thisapplication, some modifications are
needed.
We denote the projection matrix of the photograph as P, whichhas
two rows and three columns. During the learning of SOM,the results
of the forward kinematics need to be projected to a 2Dplane by P.
Here, the inverse of Jacobian matrix learned shows therelationships
between the positions of 2D plane and the joint angles.And in the
synthesis phase, the objective function is modified tominimize the
distance between the end effectors’ positions in thephoto and that
of the projection of the pose.
8 DISCUSSION AND FUTURE WORK
The problem of synthesizing natural poses that satisfy a set of
givengeometry constraints has been studied in this paper. A
methodbased on a set of reference poses and a set of matrices
learned fromexamples is presented. Our method can produce natural
looking re-sults because of two aspects. Firstly, the reference
poses with two-dimensional indices represents the space of natural
poses. Theyprovide a good initializing configuration for a given
problem. Sec-ondly, the learned matrices describe the
characteristics of motionseffectively. Because there is no
necessity to calculate the inverse ofJacobian matrix in the
synthesis phase, the time cost by computinghas been reduced
further, which can satisfy the need of time-criticalapplications.
Experiments have proven the effectiveness and ro-bustness of this
method.
However, there are some limitations in applying our
approach.Firstly, compared with the work [3], our method has to
specify theconstrained limbs at the learning phase. Although the
relationshipfor each end-effector could be learned in advance, it
requires vastcomputation and storage. Secondly, in solving
multi-constraintsproblem, the weight matrix calculated based on a
heuristic strategywill slow down the convergence rate. How to
combine the con-straints, that is, calculating the weight matrix
automatically fromexamples should be addressed in the future
work.
For future work, there are some improvements and extensionsthat
need to be considered. In the learning phase, a
two-dimensionalstructure learned from examples might not exist in
complicated mo-tions. If the dimension of SOM increases, the number
of necessarysamples increases exponentially. However, the
hierarchical SOMmight be a promising solution to this problem:
first, the high-levelSOM can be constructed as a classifier which
divides the space of
105
-
natural pose into different regions; then, a set of low-level
SOMsare constructed as the low-dimensional structure of each
region. Inthis way, we can enlarge the range of positioning
constraints. Prob-ability SOM is also an alternative, in which the
probability can beused to evaluate the result.
Inverse dynamics problem can also be considered on the basis
ofour method. We can use the force or moment instead of joint
anglesin the learning phase, and then the Jacobian matrices will
describethe relationships between the forces and the
configurations. Addi-tionally, more applications based on our
method should be exploredin future work.
ACKNOWLEDGEMENTS
We would like to thank Yanyan Yuan, Rongrong Wang, the mem-bers
of the CAS ICT virtual reality laboratory, and the anony-mous
reviewers for their advice and assistance. This workwas supported
in part by a grant from KBRPC-2002CB312104,NSFC-60403042,
NSFC-60573162, NSFC-60533070, NSFC-60603082, CISTC-2005DFA11060,
BSTPC-Z0004024040231,BNSFC-4051004, BNSFC-4062032. The data used in
this projectwas obtained from mocap.cs.cmu.edu. The database was
createdwith funding from NSF EIA-0196217.
REFERENCES
[1] E. B. Dam, M. Koch, and M. Lillholm. Quaternions,
interpola-
tion and animation. Technical report, Department of Computer
Sci-
ence,University of Copenhagen, July 17 1998.
[2] R. B. Daniel and I. Charles W. Wampler. On the inverse
kinematics of
redundant manipulators. International Journal of Robotics
Research,
7(2):3–21, 1988.
[3] K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popović.
Style-based
inverse kinematics. ACM Transactions on Graphics,
23(3):522–531,
2004.
[4] S. Haykin. Neural Networks: A Comprehensive Foundation
Second
Edition. Prentice Hall, 1999.
[5] T. Kohonen. Self-organized formation of topologically
correct feature
maps. Biological Cybernetics, 43:59–69, 1982.
[6] T. Komura, A. Kuroda, S. Kudoh, L. Tai Chiew, and Y.
Shinagawa.
An inverse kinematics method for 3d figures with motion data.
In
Computer Graphics International, pages 266–271, 2003.
[7] L. Kovar and M. Gleicher. Automated extraction and
parameteriza-
tion of motions in large data sets. ACM Transactions on
Graphics,
23(3):559–568, 2004.
[8] H. Ritter, T. Martinetz, and K. Schulten.
Topology-conserving maps
for learning visuo-motor-coordination. Neural Networks,
2:159–168,
1989.
[9] H. Ritter, T. Martinetz, and K. Schulten. Neural Computation
and
Self-Organizing Maps; An Introduction. Addison-Wesley
Longman
Publishing Co., Inc., Boston, MA, USA, 1992.
[10] C. F. Rose, P.-P. J. Sloan, and M. F. Cohen.
Artist-directed inverse-
kinematics using radial basis function interpolation. Computer
Graph-
ics Forum, 20(3), 2001.
[11] R. W. Sumner, M. Zwicker, C. Gotsman, and J. Popović.
Mesh-based
inverse kinematics. ACM Transactions on Graphics,
24(3):488–495,
2005.
[12] S. Tak and H. Ko. Example guided inverse kinematics. In
Interna-
tional Conference on Computer Graphics and Imaging, pages
19–23,
2000.
[13] D. Tolani, A. Goswami, and N. I. Badler. Real-time inverse
kinematics
techniques for anthropomorphic limbs. Graphical models,
62(5):353–
388, 2000.
[14] M. Tomohiko and K. Shigeru. Geostatistical motion
interpolation.
ACM Transactions on Graphics, 24(3):1062–1070, 2005.
[15] C. W. Wampler. Manipulator inverse kinematic solutions
based on
vector formulations and damped least squares methods. IEEE
Trans-
actions on Systems, Man, and Cybernetics, SMC-16:93–101,
1986.
[16] C. Welman. Inverse kinematics and geometric constraints for
articu-
lated figure manipulation. Master’s thesis, Simon Fraser
University,
1989.
[17] D. E. Whitney. Resolved motion rate control of manipulators
and
human prostheses. IEEE Transactions on Man-Machine Systems,
10(2):47–53, 1969.
[18] B. Widrow and M. Hoff. Adaptive switching circuits. Western
Electric
Show and Convention Record, Part 4, pages 96–104, 1960.
[19] W. A. Wolovich and H. Elliot. A computational technique for
inverse
kinematics. In Proceedings of 23rd Conference on Decision and
Con-
trol, pages 1359–1363, Las Vegas, 1984.
[20] K. Yamane and Y. Nakamura. Natural motion animation through
con-
straining and deconstraining at will. IEEE Transactions on
Visualiza-
tion and Computer Graphics, 9(3):352–360, 2003.
[21] J. Zhao and N. I. Badler. Inverse kinematics positioning
using nonlin-
ear programming for highly articulated figures. ACM Transactions
on
Graphics, 13(4):313–336, 1994.
106