Cell Tracking!

Tracking B and T cells

from 2-photon microscopy imaging

David Olivieri, Iván Gómez and Jose Faro

(University of Vigo)

www.milegroup.net

Outline

Motivation for this work

Stochastic method for tracking: SMC

• Theoretical aspects

• Some algorithm implementation details

Results:

• From simulations and animations

• From real microscopy data of 2D cell motility

Conclusions (present and future work)

Iván Gómez Conde

Immune Response

Understand complex details of immune response by understanding dynamics

Several important questions related to affinity maturation process

Cell activity and interactions

Iván Gómez Conde

Low

Affinity

Low

Affinity

High

Affinity

Activation

Produces antibodies

Motivation

Germinal centers

“Germinal centers” are the

sites of affinity maturation

Anatomic structures (in lymph

nodes) where massive

proliferation of B-cells occur

Complex interactions between

B and Th cells; spatial zones

Understanding dynamics in

gernminal center ; better

understand mechanisms of

immune response

Iván Gómez Conde

Motivation

Confocal microscopy image of GC:T-cells

blue, B-cells green

(photo courtesy of I. Wollenberb, IMM, Universidad de Lisboa

Portugal y J. Faro, Fac. Biología, Universidad de Vigo)

Dynamics

• In vivo Data:

• “2-photon Confocal microscoy” with fluorescence

excitation labelling

• Better elimination of background

Iván Gómez Conde

Motivation

Dynamics is important!

• B and T cell motility in germinal centers give

information of function

• Useful for “Inmunologic modeling” (input &

validation)

Tracking in Videos

Tracking is hard in general!

• Normally needs to be real time

• many interactions: background, camera…

• Methods: frame diff, homology, optical flow,

particle filters

What can be learned from tracking

objects?

• Tracking cells is particularly difficult

• Cells change shape, disappear, and stick to

eachother.

• complex background,

Iván Gómez Conde

Method

How to Tracking cells

Cell movement:

• Problems: Complex, overlaps, “random” component

• BUT, flourescence color is a strong feature to track

• We propose “Stochastic color based tracking”:

• SMC (stochastic monte carlo)

Iván Gómez Conde

Method

Software Components Method

Stochastic tracking

Sequential Monte Carlo (smc)

Formulate tracking as an inference problem in the context

of a Hidden Markov Model (HMM)

Observations (from

image data)

Hidden States (object

location, scale, …)

Yt Yt+1

Xt Xt+1

SMC Method

Chapman Kolmogorov Eq.

Evolution of the state (inference):

• Using the Bayesian filtering distribution:

Current Object

State

Observation

Model

Previous Object

State

Evolution

Model

SMC Method

Quantities of the Model

Prior Distribution:

• Initial distribution of object states

Evolution Model:

• How objects move between frames

Likelihood Function:

• The probability of state x given the observation y

Iván Gómez Conde

p(x0)

p(xt | xt -1)

p(yt | xt )

SMC Method

Prior distribution

User input determines the object initial position

object

Iván Gómez Conde

Initial selection

of cells by the

user

SMC Method

p(x0)

Evolution model

Evolution Model (second-

order, auto-regressive

dynamical model)

SMC Method

p(xt | xt -1)

Likelihood function

Likelihood Model (Distance metric):

Iván Gómez Conde

SMC Method

p(yt | xt )

SMC algorithm (summary)

1. Determine initial regions (roi) to track.

o From roi, store reference histogram

(each node)

2. Get image samples along trajectory of cell

o Determined from the dynamics (position,

velocity, update)

o Obtain histograms of roi; compare with

reference; keep best

3. Reorder the distribution for next sampling

Iván Gómez Conde

SMC Method

SMC Resampling

Resampling, we

change weights

Iván Gómez Conde

SMC Method

SMC Tracking pseudocode

Iván Gómez Conde

SMC Method

Results: simple animation

Showing each particle

Showing tracks of max L

Iván Gómez Conde

Results

Tracking Accuracy

Iván Gómez Conde

Results

Time Performance

Iván Gómez Conde

Results

Cells from Simulation

Iván Gómez Conde

Results

(simulation courtesy of J. Carneiro, T. Macedo, Instituto

Gulbenkian de Ciencia, Portugal)

Cells Simulation: Ambiguities

Iván Gómez Conde

Results

“unstructured” SMC leads to ambiguities

Imagine two cells sticking to each other…

Just based on color, particles will sample entire region

Not sure which cell is which after contact

Cell Ambiguities: “Present” work

Iván Gómez Conde

Results

Possible Solution: make constraints between particles

Conserve area and distance; & non-overlap condition

Node

particle Constraint

Preliminary results are promising!

Modify the Weights to include constraints

2-photon Microscopy videos

Iván Gómez Conde

Results

(videos courtesy of C.Allen, et.al

Science, 2006)

Conclusions

SMC is a promising technique for tracking cells

o Relatively easy to implement and flexible

o Can use color histogram or shape!

o Easily extended to handle 3D image stacks

o Stochastic noise can be controlled

o Present and Future:

o Extend to Constrained SMC can solve ambiguities

o Implementation of system of “constrained particles” for each node

Iván Gómez Conde

Many thanks for your

attention