Detecting & Modeling Change in Time-Varying Imagery Peggy Agouris Dept. of Spatial Information Engineering University of Maine
Detecting & Modeling Change in
Time-Varying Imagery
Peggy Agouris
Dept. of Spatial Information Engineering University of Maine
Overview
Problem(s)
Change Detection in Time-Varying Aerial Imagery
Tracking Positional Change and Modeling Spatiotemporal Behavior in Motion Imagery (incl. Video Sequences)
Examples
Problem
Change detection: one component of successful conflation of geospatial information
Integrated Environment
Translation onto new image
Object shape information (from GIS)
Shape accuracy estimates (from GIS)
Differential Snakes
Change detection
Versioning
GIS Updating
Traditional Snakes
Semi-automatic tool for object extraction
Based on the optimization of a model of
curve contrast and smoothness using
content-derived forces and an elastodynamic
model
edgecurvcontsnake EEEE ⋅+⋅+⋅= γβα
)( iedge vIE −∇=
211 2 +− +−= iiicurv vvvE
1−−−= iicont vvdE
Traditional Snakes Model
Total energy:
Continuity term:
Curvature term:
Edge term:
Optimization
Greedy algorithm : current point location is optimized, while previous and next points are fixed Stop criteria : number of points moved, change of total energy
vi
vi+1vi-1
Quality Evaluation of Extracted Road Network
Need a posteriori evaluation of object extractionResults are useful input for spatiotemporal change detectionAssumption: known energy function valuesFor sample points along an extracted object, values of uncertainty are generated using fuzzy rules
Quality Evaluation Rules
High Medium Low
Low High Low Low
High High Medium High
DEE
Points of interest are determined based on statistical properties of energy
Fuzzy rules of the form :
• If Et is LOW and DEt is LOW then U is LOW
d
Fin
Fout
vi
v0i
uncedgecurvcontsnake EEEEE ⋅+⋅+⋅+⋅= δγβα
Additional energy term (uncertainty)
Action is similar to an elastic spring force
Differential Snake Model
Uncertainty Energy
dvUncD
Eii
unc ⋅⋅
=)0(
1
dDi
1/Unc(v0i)
0
Eunc
Change Detection vs. Versioning
Change is detected if a road segment has moved beyond the stochastic range of older information
Versioning identifies road segments that can be delineated in the new image with better accuracy than their current database record
Example
Prior (red) and current (blue) road shape information
Change Detection & Versioning Experiments
Buffer zones of influence of prior
information
Result of change detection (blue line)
Result of versioning (purple line)
Change Detection & Versioning Experiments (cont.)
Prior and current road shape information
Buffer zones of influence of prior
information
After change detection (blue)
and versioning (purple)
1
2
3
4
5
6
1
2
3
4
5
6
Differential Snakes GUI (Change Detection & Versioning)
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
Performance and Accuracy Issues
Typical Performance Metrics
Average values for a road segment spanning
a 512x512 image window:
• Change Detection: 2.095 sec
• Versioning: 0.561 sec
Change Detection for
Closed Contour Objects
Important tool for dynamic scene analysis
Applications: surveillance, environmental, transportation, biomedical, etc.
Quick and efficient, requires proper initialization, assumes frequent monitoring (small movement of object between frames)
Differential Snakes for Tracking Object Contours
Extracted Object Contour from
Previous Frame
New Frame Information
Differential Snakes
New Object Contour
Estimation of Translation and Rotation
Translation: difference of positions of two geometric centersRotation: difference of direction of principal axes
Δϕ
Estimation of Uniform Expansion
Ratio of areas = (ratio of perimeters)2
Estimation of Radial Deformation
Use of polygon clipping techniques: - calculate the intersections between two input polygons- label edges as inside, outside, or shared- find the minimal polygons which are created by intersection- classify all minimal polygons into the output sets A∩B, A/B, and A\B
Experiments with Moving Objects
Track changes in the shape of an object
Example: a liquid that deforms non-uniformly
We show five distinct frames and the detected
change between them (frames n, n+1)
Area threshold to ignore small polygon changes
Integration of spatiotemporal tracking process in
a GUI (in Matlab)
Experiment with a Moving Object
100 150 200 250 300 350 400 450 500
100
150
200
250
300
350
400100 150 200 250 300 350 400 450 500
100
150
200
250
300
350
400100 150 200 250 300 350 400 450 500
100
150
200
250
300
350
400100 150 200 250 300 350 400 450 500
100
150
200
250
300
350
400
GUI for Spatiotemporal Change Detection
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
Remarks
Integration of object extraction and change detection
Introduction of uncertainty as external energy in a deformable modelChange detection using the uncertainty of the extraction
Framework for spatiotemporal tracking of object deformations
Estimate translation, rotation, radial deformations using geometric properties
Positional Change Detection & Modeling
in Motion Imagery
Problem
Detecting change in position and shape/extent of objects or events across time and space Modeling their spatiotemporal behavior
Trends in imagery collection: from static to motion and from single to multiple sensors.
Tremendous amounts of data.
Bottleneck in the analyst workforce.
Rationale
Automated motion imagery analysis solutions
Automation at various levels of the analysisprocess:
automated identification of trajectories in single video feeds (i.e. tracking positional change over time)automated content analysis to identify interesting spatiotemporal activities and support queries
Needs
Motion Trajectory Identification
Nodal Representation of Trajectories
Spatiotemporal Helix Modeling
Essential Issues
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
0
200
400
801001201401601802002202400
10
20
30
40
50
60
70
80
90
x (column pixels)
1
3
2
55
1
3
2
5
2
3
1
5
2
5
3
5
1
3
7
2
5
1
7
3
5
2
6
7
1
3
4
2
6
1
7
3
4
2
6
8
3
4
1
2
6
8
3
4
6
2
1
8
3
4
6
2
1
Moving Object Trajectories
8
3
4
6
2
1
4
2
y (row pixels)
6
1
66
6
fram
e (t
ime)
Modeling Spatiotemporal Change
Over time objects/events may change their:
location (movement)
outline (deformation)
Need:
an integrated representation of movement
and deformation
Trajectories of moving objects:
3-d collections of points evolving through S-T space
•Generalization
•Summarization
•Behavior Analysis
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.QuickTime™ and a
decompressorare needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Moving Objects in the SpatioTemporal Domain
YUV420 codec decompressorare needed to see this picture.
YUV420 codec decompressorare needed to see this picture.
x
y
t
x
y
t
Summarization
The SpatioTemporal HelixAn integrated representation of movement and
deformation, and
A signature of an object’s spatiotemporal behavior.
Comprises a spine and prongsSpine models trajectory• Nodes: acceleration, deceleration, rotation
Prongs express deformation• Changes of a predefined magnitude• Recorded as time, percent change, azimuth
Helix RepresentationSpine:Spine: expresses spatioexpresses spatio--
temporal 3temporal 3--D movement of D movement of
the center of mass.the center of mass.
Prongs:Prongs: express expansion or express expansion or
collapse of the objectcollapse of the object’’s outline s outline
The Helix as a Spatiotemporal Database Index
Helixobjidt1,t2={node1,…noden; prong1,..prongm}
•• Node:Node: nnii(x,y,t,q(x,y,t,q))
•• Prong: Prong: ppjj(t,r,a(t,r,a11,a,a22))
Collecting Spine & Prong Information
Two novel image analysis techniques:
SOM with geometric analysis (g-SOM)•Describes ST trajectory of center of mass
Differential snakes•Allows calculation of percent change in
outline
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
Generalization
Hurricane Frances
Hurricane Charley
Hurricane Helixes
Hurricane Helixes: Charlie
QuickTime™ and aCinepak decompressor
are needed to see this picture.
QuickTime™ and aCinepak decompressor
are needed to see this picture.
Hurricane Helixes: Frances
QuickTime™ and aCinepak decompressor
are needed to see this picture.
QuickTime™ and aCinepak decompressor
are needed to see this picture.
Concluding Remarks
Incorporating uncertainty in change detection improves conflation and eliminates false positivesDetection of positional and shape change in motion imagery contributes to a better understanding of behavior of evolving events
For more information:
Peggy AgourisAnthony Stefanidis
{peggy, tony}@spatial.maine.edu
http://dipa.spatial.maine.edu