Mining and Understanding Events in Crowd Scenesmedialab.sjtu.edu.cn/teaching/CV/Lec/Lec10-network... · "A diffusion and clustering-based approach for finding coherent motions and

Mining and Understanding

Events in Crowd Scenes

Weiyao Lin (林巍峣)

Shanghai Jiao Tong University

Oct. 13. 2016

Background

Issues in crowd scene understanding:

• Semantic Region Segmentation Crowd Activity Recognition

• Unsupervised Recurrent Motion Pattern Mining

• Cross-view Motion Pattern Matching

Background Related works

• Trajectory-based methods

• Trajectory (or tracklet) extraction in crowed scenes

become difficult and unreliable

• Low-level feature based methods

• Have limitations in achieving precise motion flow

patterns under scenes with complex motions

Our Approach

• Parsing crowd scenes based on coherent motion regions

Framework

• Coarse-to-Fine Thermal Diffusion Better motion field:

Thermal Energy Field (TEF)

• Two-Step Clustering Finding reliable semantic regions

• Cluster and merge Finding recurrent motion patterns

Coherent Motion Region Detection Coarse-to-fine Thermal Diffusion Process

Thermal diffusion equation: The final diffused thermal

energy for P after l sec:

The final individual thermal energy from Q to P:

Coherent Motion Region Detection

Framework

• Coarse-to-Fine Thermal Diffusion Better motion field:

Thermal Energy Field (TEF)

• Two-Step Clustering Finding reliable semantic regions

• Cluster and merge Finding recurrent motion patterns

Finding Semantic Regions

Two step Clustering

• Step 1: Cluster coherent motions

• Similarity between coherent motions

where

Finding Semantic Regions

Two step Clustering

• Step 2: Cluster to find semantic regions

Framework

• Coarse-to-Fine Thermal Diffusion Better motion field:

Thermal Energy Field (TEF)

• Two-Step Clustering Finding reliable semantic regions

• Cluster and merge Finding recurrent motion patterns

Finding recurrent motion patterns Cluster and Merge Process

• Step 1: Iterative frame-level clustering

Finding recurrent motion patterns

Similarity between frames

Joint similarity for matched

coherent regions

Joint similarity for unmatched

coherent regions

Finding recurrent motion patterns Cluster and Merge Process

• Step 2: Iterative frame-level clustering

• Step 3: Flow curve extraction

Cross-view Motion Pattern Matching

Define matching cliques

Find best matching by optimizing clique cost

Experimental Results

(a): Ground Truth, (b): Our approach, (c): CVPR’07, (d): IEEE Trans. SMC’ 12, (e):

ECCV’12, (f): CVPR’13, (g): ECCV’10, (h) CVIU’13

Experimental Results

Semantic region detection

Experimental Results

Recognizing crowd activities

Dense

Trajectory

Experimental Results

Mining Recurrent Motion Patterns

Experimental Results

Matching cross-scene motion patterns

References

[1] Weiyao Lin*, Y. Mi, W. Wang, et al. "A diffusion and clustering-based

approach for finding coherent motions and understanding crowd scenes,"

IEEE Trans. Image Processing, vol. 25, no. 4, pp. 1674-1687, 2016.

[2] W. Wang, Weiyao Lin* et al., "Finding coherent motions and semantic

regions in crowd scenes: a diffusion and clustering approach," ECCV, 2014.

[3] Weiyao Lin*, Y. Mi et al., "Finding coherent motions and understanding

crowd scenes: a diffusion and clustering-based approach," CVPR Scene

UNderstanding workshop, 2015.

[4] L. Liu, Weiyao Lin*, et al., "Traffic flow matching with clique and triplet

cues," MMSP, 2015.

For more information, please visit my personal

webpage:

• http://wylin2.drivehq.com/

Thanks！