ICPR 2012

Foreground Detection via Robust Low Rank MatrixFactorization including Spatial Constraint with Iterative

Reweighted Regression

C. Guyon, T. Bouwmans and E. [email protected]

MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, FrancePresenter: Muriel Visani (L3i lab - University of la Rochelle)

—ICPR2012, Tsukuba, Japan

November 14, 2012

C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 1 / 25

Summary

1 Introduction and motivation on IRLS

2 Temporal constraint with an adapted norm

3 Diagram flow and spatial constraint

4 Experimental Results

5 Conclusion


Introduction and motivation

Purpose

Foreground detection : Segmentation of moving objects in video sequenceacquired by a fixed camera.

Background modeling : Modelization of all that is not moving object.

Involved applicationsSurveillance cameraMotion capture

On the importanceCrucial Task : Often the first step of a full video surveillance system.

Strategy usedEigenbackground decomposition.


Eigenbackgrounds

Find an « ideal » subspace of the video sequence, which describes the bestas possible the (dynamic) background.

Fig.1 The common process of background subtraction via PCA (Principal ComponentAnalysis). At the final step, an adaptative threshold is used to get a binary image.

Without a robust framework, the moving object may be absorbed in the model !C. Guyon, T. Bouwmans and E. Zahzah [email protected] (MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France Presenter: Muriel Visani (L3i lab - University of la Rochelle) — ICPR2012, Tsukuba, Japan )Foreground Detection via Robust Low Rank Matrix Factorization including Spatial Constraint with Iterative Reweighted RegressionNovember 14, 2012 4 / 25

Data Structure Transformation

First, we consider a video sequence as a matrix A ∈ Rn×m

n is the amount of pixels in a frame (∼ 106)m is the number of frames considered (∼ 200)


IRLS : Vector version (1)

The usual IRLS (Iteratively Reweighted Least Squares) scheme for solveargmin

x||Ax− b||α is given by :

D(i) = diag((ε+ |b − Ax (i)|)α−2)x (i+1) = (AtD(i)A)−1AtD(i)b

(1)

This IRLS method is convergent for 1 ≤ α < 3. An more suitableformulation is :

r (i) = b − Ax (i)

D = diag((ε+ |r (i)|)α−2)y (i) = (A′DA)−1A′Dr (i)

x (i+1) = x (i) + (1+ λopt)y(i)

(2)

for λopt computed in a second inner loop. It is convergent for 1 ≤ α < +∞


IRLS : Vector version (2)

For spatio/temporal RPCA, it needs to solve the following general problem :

argminx

||Ax − b||α + λ||Cx − d ||β (3)

By derivation, the associated IRLS scheme is,

r1 = b − Ax (i), r2 = d − Cx (i), e1 = ε+ |r1|, e2 = ε+ |r2|D1 = (

∑eα1 )

1α−1diag(eα−21 ),D2 = λ(

∑eβ2 )

1β−1diag(eβ−22 )

y (i) = (A′D1A + C ′D2C)−1(A′D1r1 + C ′D2r2)

x (i+1) = x (i) + (1 + λopt)y(i)

(4)

Good news : Just few lines in Matlab !


IRLS : Matrix Version

More generally, we consider the following matrix regression problem withtwo parameters norm (α, β) and a weighted matrix W ,

minX||AX − B||α,β

W

with ||Mij ||α,βW

= (n∑

i=1

(m∑j=1

Wij |Mij |β)αβ )

1α (5)

The problem is solved in the same manner on matrices with a reweightedregression strategy,

Until X is stable, repeat on each k-columnR ← B − AXS ← ε+ |R|Dk ← diag(Sβ−2ik ◦ (

∑j(S

βij ◦Wij))

αβ−1 ◦Wik)k

Xik ← Xik +(1+Λ(max(α, β)))(AtDkA)−1AtDkRik

(6)


Various RPCA formulation (only for α = 1)

PCA with a fixed rank is : minL,S

||S ||Fs.t. Rank(L) = k

A = L + S

(7)

R(obust)PCA is (Non convex and NP-hard) :

minL,S

||σ(L)||0 + λ||S ||0s.t. A = L + S

(8)

Convex relaxed problem of (8) is RPCA-PCP proposed by Candès et al. [1] :

minL,S

||σ(L)||1 + λ||S ||1s.t. A = L + S

(9)

where σ(L) means singular values of L.A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) :

minL,S

||σ(L)||1 + λ||S ||1s.t. ||A− L− S ||F < δ

(10)

All of them could be solved by Augmented Lagrangian Multipliers (ALM).


Video examples

Some examples, temporal RPCA and ideal RPCA with ground truth fitting.


Summary





5 Conclusion


Sparse solution

In RPCA, residual error is sparse.Using the RPCA decomposition on a synthetic low-rank random matrixplus noise, the error looks like :

Same principle with video. Sparse noise (or outliers) are the moving objects.


Let’s play with norms

Varying the α, β norm → Different kind of recovering pattern error.


Let’s play with norms...(2)

Some issuesWhat is the best specific norm for temporal constrain ?Initial assumption is ||.||2,1. Confirmed experimentally ?


Validation

If ideal eigenbakgrounds are that, best norm should be ...

Let us denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCA


Experimental validation

Let us denote Lα,β , the low-rankrecovered matrix with a ||.||α,β-PCA.The plot shows the error between||Lopt − Lα,β||F for parameters αand β chosen freely. The darkestvalue means that the error is thesmallest here.

||S ||2,1 is not optimal, but for convenience we use it.The benefit of the ad hoc block-sparse hypothesis is confirmed bytesting its efficiency directly on video dataset.

Experimentation done on dynamic category of dataset change detectionworkshop 2012 : http://www.changedetection.net/


http://www.changedetection.net/

Summary





5 Conclusion


Overview & addition of a spatial constraint via TV

Figure: Overview of the learning and evaluation process. Learning process needsGT (Ground Truth) for better fits the eigenbackground components.

Spatial Constraint via TVSuppose A = L+ S where L and S are computed via some kind ofRPCA techniques with the addition of Total Variation penalty on S .This penalty increases connected (or connexe) shapes.


Exemple with a synthetic 1-D signal


Summary





5 Conclusion


Experimental Protocol

RPCA-IRLS is compared for the following four recent robust methods :

Low-Rank Block sparse Decomposition (LBD, 2011) [3]

Low-Rank Representation (LRR, 2011) [4]

Symmetric Alternating Direction Augmented Lagrangian (SADAL, 2011) [5]

Grassmannian Robust Adaptive Subspace Tracking Algorithm (GRASTA, 2012) [6]

References[1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM,

vol. 58, no. 3, May 2011.

[2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISITProceedings, pp. 1518-1522, Jun. 2010.

[3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrixdecomposition, CISS 2011, 2011.

[4] Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rankrepresentation. NIPS 2011, Dec. 2011.

[5] S. Ma. Algorithms for sparse and low-rank optimization : Convergence, complexity and applications. Thesis,2011.

[6] J. He, L. Balzano, and A. Szlam. Incremental gradient on the grassmannian for online foreground andbackground separation in subsampled video. Conference on Computer Vision and Pattern Recognition(CVPR), June 2012.


Experimental Protocol & Quantitative Results

Optimal threshold is chosen for maximizing F-measure criterionwhich is based 2× 2 histogram of True/false/positive/negative :

DR =TP

TP + FN, Prec =

TP

TP + FP, F =

2 DR Prec

DR + Prec

Good performance is then obtained when the F-measure is closed to 1Time consumption is not take into account in the evaluation process.

Figure: F-Measure on the Wallflower and I2R dataset.


Quantitative Results

Here, we show other experimental results on the real dataset of BMC 2012,

Video Recall Precision F-measure PSNR Visual Results1 0.9139 0.7170 0.8036 38.24252 0.8785 0.8656 0.8720 26.77213 0.9658 0.8120 0.8822 37.70534 0.9550 0.7187 0.8202 39.36995 0.9102 0.5589 0.6925 30.58766 0.9002 0.7727 0.8316 29.99947 0.9116 0.8401 0.8744 26.83508 0.8651 0.6710 0.7558 30.50409 0.9309 0.8239 0.8741 55.1163

Table: Quantitative results with common criterions. Last column : sample of theoriginal video, GT and our results of the first four real video sequences.


Summary





5 Conclusion


Conclusion

Advantages

Experiments on video surveillance datasets show that this approach is morerobust than other recent RPCA formulation in presence of dynamicbackgrounds (DC) and illumination changes (IC).

Well suited for video with spatially spread and temporarily sparse outliers.

Disadvantages

Small local motions, like « waving trees » are not (yet) well modelized bythis kind of global PCA. For example, IC needs few eigenBackground andDC needs more with the risk to integrate moving objects into the model.

Future WorksLack in computation time : Further research consists in developping anincremental version to update the model at every frame and to achieve thereal-time requirements.


ICPR 2012

Technology

matrix version

d cx i

d i a1

xed rank

robust framework

y i good news

argmin ax b

weighted matrix w