Animating a Three Dimensional Whole Body Point Scan

Animating a Three DimensionalWhole Body Point Scan

Pushkar Deshmukh

Computer Science Department

Clemson University

July 1999

Outline• Background and Motivation • Animation Process

– Basic Idea

– Software Architecture

– Extended Free Form Deformation

– Designing Lattices

– Skeleton and Joint Structure

– Generating Animations

• Results• Applications and Future Work

3D Whole Body Scanners

Cyberware Whole Body 3D scanner

4 cameras 120K+ points17-sec. Scan x, y, z, RGB values

3D Whole Body Scanners

[TC]2 Whole Body 3D scanner

6 cameras 350K+ points2-sec. Scan x, y, z values

3D Whole Body Scan

3D Whole Body Scan

Close-up view

Motivation

• Ergonomics– Study of relation between workers and their

environment– Used for product design– Faster and Cheaper design process– Measurements from 3D scan

• 3DM Project

– Animating 3D scan

3D Scan Animation

One Approach: Segmenting Scan

Jack.gif Brian.gif Animation.gif

Free Form Deformation

• Basic Idea– Embed an object in a

deformable region

– Deform the region to deform the object

• Developed by Thomas Sederberg and Scott Parry (Free-Form Deformation of Solid Geometric Models. SIGGRAPH '86, volume 20, pages 151-160, ACM, August 1986.)

• Lattices

U

V

W

Parallelepiped Lattice


• Mathematical Definition– Tensor product trivariate Bernstein polynomial

where Lijk is the (i, j, k)th control point of the lattice, (u,v,w) are lattice space coordinates of point P and the lattice has l+1, m+1, n+1 planes in the U, V, W directions respectively.

m

j

kknn

k

jjmiill

i

wwk

nvv

j

muu

i

lzyx

0ijk

00

L111,,P

Free Form Deformation• Steps

– Calculate lattice space coordinates (u, v, w) (Freeze lattice)

– Deform the lattice

– Deform object

Example of FFD


• Limitation– Only parallelepiped lattices

• Extended Free Form Deformation– developed by Coquillart (Extended Free-Form Deformation: A

Sculpturing Tool for 3D Geometric Modelling. Computer Graphics, Volume 24, Number 4, pages 187-196, August 1990, ACM SIGGRAPH)

– allows non-parallelepiped lattices

Animation Process

Basic Idea• Enclose scan data

points in EFFD lattices• Freeze lattices• Move lattice control

points• Deform scan to

generate animations

Steps in Animation Process

• Read a 3D scan

• Build EFFD lattices around the scan

• Build skeleton and joint structure

• Freeze the lattices

• Generate postures by varying joint angles

• Interpolate between postures to generate animation

Software Architecture

User Interface

Skeleton & Joint Structure

Extended Free Form Deformation

3D Human Scan

Software Modules

CMainAnim

CMatrix

CDataPoint

CLattice CJoint

CSkeleton

CComplexJoint

Class diagram


P(x, y, z)

X

Y

Z

Start: A point P(x, y, z) enclosed in a lattice

P(x', y', z')

X

Y

Z

Second Step: Find new (x, y, z) coordinates of P using its (u, v, w) coordinates and the lattice control points.

P(u, v, w)

U

V

W

First Step: Calculate (u, v, w) coordinates of P

Find Cartesian Coordinate

m

j

kknn

k

jjmiill

i

wwk

nvv

j

muu

i

lzyx

0ijk

00

L111,,P

Where, - Lijk is the cartesian coordinate of the (i, j, k)th control point of the lattice - (u, v, w) is the lattice space coordinates of point P - the lattice has l + 1, m + 1, and n + 1 planes in the U, V, and W directions respectively and

!!

!

!!

!

!!

!

kkn

n

k

n

jjm

m

j

m

iil

l

i

l

Find Lattice Space Coordinate

Lattice Space

U

W

V

P(ui, vi, wi)

Cartesian Coordinate Space

X

Z

Y

P(xi, yi, zi)

P(xf, yf, zf)

VL000

L100

L101

L001

L011L111

L110L010

FFD Equation

-1iJ

~

• Iterative Process– Aim, find (u, v, w) coordinate for point P(xf, yf, zf)

– Start with point P(u0 = 0.5, v0 = 0.5, w0 = 0.5)

– Find cartesian coordinate P(x0, y0, z0)

Find Lattice Space Coordinate

– We obtain

where is inverse of transpose of local approximate of Jacobian at (u0, v0, w0)

– In general

– Terminating condition

-10000f000111 J

~,,P,,P,,,, zyxzyxwvuwvu ff

-10J

~

-1if111 J

~,,P,,P,,,, iiiffiiiiii zyxzyxwvuwvu

JacobianSuppose we have a function

where X is of dimension n and L is of dimension m, then

dX = J(L) dL

and, the (i, j)th element of J is given by

From above equation

LX f

j

i

l

f

ijJ

-1-1 J~

XXJL ddd

Jacobian

wvuzw

wvuzv

wvuzu

wvuyw

wvuyv

wvuyu

wvuxw

wvuxv

wvuxu

,,,,,,

,,,,,,

,,,,,,

J

kjikjikji

kjikjikji

kjikjikji

zzyyxx

zzyyxx

zzyyxx

,,1kj,i,,,1kj,i,,,1kj,i,

,,k1,ji,,,k1,ji,,,k1,ji,

,,kj,1,i,,kj,1,i,,kj,1,i

J~

Using forward differences to obtain an estimate of Jacobian at lattice control points we have:

Jacobian

• Estimate jacobian at point P(u, v, w) using trilinear interpolation

• Extend the jacobian

Terminating the iterative process

• V = P(xf, yf, zf) - P(xi, yi, zi) <

• Number of Iterations (NI) and0 < NI < 50 = 0.0001

50 < NI < 500 = 0.01

NI = 500 = 0.1

• Average number of iterations ~ 4


• Point-in-lattice membership

P(x, y, z)

X

Y

Z

Start: A point P(x, y, z) enclosed in a lattice

P(x', y', z')

X

Y

Z

Second Step: Find new (x, y, z) coordinates of P

P(u, v, w)

U

V

W

First Step: Calculate (u, v, w) coordinates of P

Designing Lattices

• Five EFFD lattices• Each lattice has

– Two planes in X direction

– Two planes in Y direction

– Seven planes in Z direction

XYZ Plane 0

Plane 6

Designing Arm and Leg Lattices

Plane 0

Plane 6

Plane 0

Plane 6

Plane 3

Plane 3

Designing Torso Lattice

Plane 0

Plane 6

Plane 2

Plane 4

Designing Lattices

Designing Lattices

• Moving Control Points

• Maintaining Continuity

• Avoid Overlap

Skeleton and Joint Structure

• Need– Deform the lattice control points in a controlled manner

• Provides– Ability to move the control points by specifying joint

angles

Skeleton and Joint Structure

CenterRight Hip

Right Knee

Left Hip

Left Knee

Spine

NeckBase

Left Shoulder

Right Shoulder

Left Elbow

Right Elbow

Right Hip

Spine

Left Hip

Right Knee Left Knee

Neck Base

Right Shoulder Left Shoulder

Right Elbow Left Elbow

Root Position

Center

Joint Dependency HierarchySkeleton and Joint Structure

Associating Joints with Lattices

• Link to lattices– Using lattice plane numbers

• Control points affected by the joint

• Additional Constraints– Proper placement of lattice

planes

Plane 0

Plane 6

Plane 3

Joints and Lattice Control Points

• Joint local coordinate system

• Local angle of rotations

• Sequence of rotation– Z, X, and then Y

X

Y

Z

X

Y

Z

X

Y

ZY

X Z

Y

XZ

Y

XZ

Y

XZ

Y

X

Z

Y

X

Z

Y

X

Z

Y

X

Z

Joint local coordinate system

Joints and Lattice Control Points• Local coordinates of the control points• Changing local angle of rotation

Y

X Z

Y

XZ

P

Y

X

ZY

XZ

P

Initial state Rotating shoulder about X axis

Joints and Lattice Control Points

• Find initial joint angles

Generating Animations

• Postures

Initial posture A posture with hands raised and upper body turned

Generating Animations

• Saving posture files

• Animations generated by linear interpolation between postures

ROOT_POSITION 0.010000 -0.050000 0.000000CENTER 0.000000 0.000000 0.000000SPINE 0.000000 0.000000 0.000000NECK_BASE 0.000000 0.000000 0.000000NECK 0.000000 0.000000 0.000000RIGHT_ELBOW 48.366463 25.538908 0.000000RIGHT_SHOULDER -50.856022 0.000000 0.000000RIGHT_HIP -2.726310 -2.723229 0.000000RIGHT_KNEE -4.720062 5.182696 0.000000LEFT_ELBOW 36.594429 -25.992308 0.000000LEFT_SHOULDER -49.398701 0.000000 0.000000LEFT_HIP -0.000002 2.862404 0.000000LEFT_KNEE -9.512845 -7.522387 0.000000

Sample posture file

Results

Upper body movement

Results

Movement of lattice control points

Results

Leg Movement

Results

Handshake

Applications

• Improving ergonomics in product design

• Visualizing simulations involving human characters

• Computer games

• Designing safer and comfortable apparel

• Representing human participants in VR

• Film industry

Conclusion

• Simple animations can be created from a 3D human scan

• The 3D scan works as a good human model

• Further improvement in term of speed and quality of animation are possible

Future Work

• Improving speed of the process– Reduce number of data points– Automate lattice designing– Using accelerated search algorithm for point-in-

lattice membership– Providing multi-processor support

Future Work

• Improving quality of generated animations– Implementing effects like squash and stretch of

muscles– Increasing number of joints– Restricting the degree of freedom and adding

limits on magnitude of rotations– Using better interpolation algorithms– Interfacing with human simulation packages

Acknowledgements

I would like to thank my committee members

Dr. Roy Pargas (Committee Chair), Dr. Andrew Duchowski and Dr. Gene Tagliarini

for their guidance and for their valuable and insightful criticism and advice.

Animating a Three Dimensional Whole Body Point Scan

Documents