Segmentation and Tracking of Ionospheric Storm Enhancements

Segmentation and Tracking of Ionospheric Storm Enhancements

Matthew P. Foster & Adrian N. EvansUniversity of Bath

SPIE Europe Remote Sensing 2008

Contents I

Basics Segmentation Tracking

•Aims and objectives

•The Ionosphere & geomagnetic storms

•Storm enhancements & data

Contents II

Basics Segmentation Tracking

•Attribute mathematical morphology

•Ground-truth generation & temporal feedback

•Splitting features & segmented outputs

Contents III

Basics Segmentation Tracking

•Motion from boundaries using shape contexts

•Depletion issues

•Vector outputs

Basics

Aims & Objectives•Geomagnetic storms

cause ionospheric enhancements

•We want to track these as they cross the northern polar region

•This is hard – the images are tiny!

http://www.flickr.com/photos/orvaratli

The Ionosphere•Atmospheric region from

50 to over 1000 km

•Electrons and ions can exist for short periods

•They form layers and affect radio propagation

•Electron density is largely determined by the Sun

Public domain image from WikiMedia Commons

Geomagnetic StormsB

ow

Shock

Mag

net

opau

se

Magnetotail

Reconnection

Reconnection

Solar wind

•When large amounts of material are ejected by the Sun

•Some can be injected into the Ionosphere

•This is a geomagnetic storm

Storm Enhancements

• The injected material increases the number of electrons in the ionosphere

• These increases are called storm enhanced density and form a tongue which moves over the northern polar cap

• It’s this we want to track

Data from GPS •Raw data from MIDAS

(tomographic imaging software)

•100˚ x 100˚ grid

•4˚ resolution

•25 x 25 pixels

•5 minutes between frames

Data from GPS •Raw data from MIDAS

(tomographic imaging software)

•100˚ x 100˚ grid

•4˚ resolution

•25 x 25 pixels

•5 minutes between frames

USA

Scandinavia

Russia

Greenland

Canada

Direction of Incident Solar Radiation

Tongue of Ionisation

Sunlight

Traditional Approaches

• Optical flow requires greater consistency

• Block matching requires texture

• Correspondence methods require feature detection (and filters are too large)

• So we decided to opt for a two-stage approach (segmentation & tracking)

Segmentation

Attribute Morphology•Greyscale mathematical

morphology – extended to support general attributes

•Such as area, contrast, moments etc…

•Contrast closing is useful in this case as area parameter is too sensitive

•Removes background

1

23

Attribute Morphology•Greyscale mathematical

morphology – extended to support general attributes

•Such as area, contrast, moments etc…

•Contrast closing is useful in this case as area parameter is too sensitive

•Removes background

1

23

Ground-truth Generation•A contrast value is

needed for the opening operation

•We hand-segmented the sequence

•And tested different values to choose the best overall (33)

Ground-truth Generation•A contrast value is

needed for the opening operation

•We hand-segmented the sequence

•And tested different values to choose the best overall (33)

Input

Increase

contrast

Segment

Check area

against

previous

Ouput

Decrease

contrast

Temporal Feedback

•The segmented results are fairly good

•But changes between frames mean that they lack consistency

•Temporal feedback helps here

Splitting Features

• Watershed segmentation can be used to split features joined by saddle shaped features

• Invert segmented images and apply watershed

Segmented Outputs• Outputs from feedback show better spatiotemporal

consistency

• Boundary tracing used to convert to smoothing splines

• Ready for estimating motion using shape correspondence

20:00 20:50 21:40

Tracking

Motion from Boundaries

• Motion can be estimated by calculating boundary correspondences

• This can be done by misusing shape contexts

• i.e. don’t compute similarity metrics

• Which map one boundary into another

• Subtracting coordinates of corresponding points gives motion vectors

Shape Contexts I

•For each boundary point

•Subtract coordinates from the others

•Convert to polar form

•Bin to create a 2-D histogram

Ref: S. Belongie, et al. Shape matching and object recognition using shape contexts. Transactions on Pattern Analysis and Machine Intelligence, 24(4):509–522, Apr 2002.

Context n

Boundary 1 Boundary 2

Overlaid Boundaries

Boundary Motion

Shape Contexts II•Compare stacks of 2-D

histograms using χ², EMD or diffusion distance

•Use bipartite matching to get correspondence

•Subtract corresponding coordinates to get vectors

20:00 20:05 20:10

Vectors & Depletion Issues• Gives good results, especially near beginning

• A few frames have vectors pointing the wrong way!

• This happens because of depletion

• Electron densities drop at night (top right)

20:00 20:05 20:10

Vectors & Depletion Issues• Gives good results, especially near beginning

• A few frames have vectors pointing the wrong way!

• This happens because of depletion

• Electron densities drop at night (top right)

21:25 21:30 21:35

Vector Outputs• The depletion problem can be fixed by

detecting the occurrences

• And replacing the vectors with the previous frames – interpolated to the correct positions.

20:00 20:50 21:40

• Vector fields can be made more useful by making them more uniformly available

• Interpolating / regularising them across the object increases usefulness

Increasing Vector Density

20:50 21:40 22:30

• Vector fields can be made more useful by making them more uniformly available

• Interpolating / regularising them across the object increases usefulness

Increasing Vector Density

20:50 21:40 22:30

Conclusions• Attribute morphology can be useful for

segmentation where area or others are too sensitive

• Shape context matching can be used to infer boundary motion

• This methodology allows some analysis despite very low image resolution

• Future work will focus on additional data and other storm events

[email protected]

Extra Slides

a ab

Apparent Boundary Motion

Actual Flow Direction

Extra Slides

Connected Component GraphArea 3 Opened Image

B ACE F

GIH J

D

Set to be removed from output imageX

J

H

E

B

I

G

D

A

F

C

Extra Slides

Connected Component GraphContrast 2 Opened Image

B ACE F

GIH J

D

Set to be removed from output imageX

J

H

E

B

I

G

D

A

F

C