Dynamic Data Analysis Projects in the Image Analysis and Motion Capture L abs

Dynamic Data Analysis Projects in the Image Analysis and Motion Capture Labs

Figure: functional brain MRI of a monetary reward task; left: 16 cocaine subjects, more connections in the cerebellum (green); right: 12 control subjects, more connections in the prefrontal cortex(red)

Local constancy:

Figure: local constancy in a 2D dataset (spatial neighborhood in black dashed lines) - local constancy does not discourage long range

interactions

Strictly Concave Penalized Maximum Likelihood:We perform maximum likelihood estimation with sparseness and local constancy priors

log-likelihood of the dataset sparsenesspenalty

local constancy

penalty regularization parameters

precision matrix for N variables sample covariance matrix discrete derivative operator for M spatial neighborhood relationships

Problem definition

The deformation error can be measured in the 2D domain using conformal mapping and three correspondences, leading to a high-order graph matching problem

In dynamic 3d data non-rigid registration is essential for 3d surface tracking, expression analysis and transfer, dense motion capture data processing etc.

Current registration results:

Non-rigid Surface Registration Using High-Order Graph MatchingSparse and Locally Constant Gaussian Graphical Models

Figure: manually labeled walking sequence. Right: leg/leg, hand/leg interaction in red, independent leg motion in blue

Figure: cardiac MRI displacement and the corresponding spatial

manifold

0.6

0.7

0.8

0.9

1

1.1

Rel

ativ

e lo

g-lik

elih

ood

Synthetic CardiacMRI

Walkingsequence

Brain MRICocaine

Brain MRIControl

* **

Indep MB-and [1] MB-or CovSel [2] GLasso [3] SLCGGM

Figure: cross-validated log-likelihood on the testing set*not statistically significantly different from our method

Results

3 Example Applications:

Simultaneous Analysis of Facial Expression and EEG DataGoal: Examine facial expressions related to drug craving and drug addictionDataset: Videos of the facial expressions of subjects and simultaneously captured EEG (electroencephalogram) dataThe subjects watch a series of images belonging in several categories (happy, unpleasant, drugs, neutral)Method: -Facial expression features are tracked using an Active Appearance Model (AAM)- FACS (Facial Action Coding System) codes are retrieved from the feature movement

Statistical Shadow and Illumination Estimation for Real-World ImagesGoal: Estimate the Illumination environment from a single image, with rough knowledge of the 3D geometry and in the presence of texture

A novel cue for shading/shadow extraction

An MRF model for robust illumination estimation-Models the creation of cast shadows in a statistical framework- Allows estimation of the illumination from real images, modeling objects with bounding boxes or general class geometry

Results

Illumination from Caltech 101 motorbike images, using a common 3D model for the whole class:

Applications: integration in scene understanding, search in large image databases, augmented reality etc

Goal: We want to explore the structure of probabilistic relationships in massive spatiotemporal datasets. We want to learn sparse Gaussian graphical models, while enforcing spatial coherence of the dependence and independence relationships. Such learned structure permits efficient inference but also gives insights into the nature of the data

Challenges: - original data are not registered in object space and the points may have different motion vectors and velocities)- The large size of the datasets (tens of thousands of 3d points per frame) require accurate and efficient processing

Examples

+ =

Expression transfer:

Tracking subtle details:

NSF I/UCRC Workshop