The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh.

The role of contours in form-The role of contours in form-based motion processingbased motion processing

P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh

Form from motion orStructure from motion

Form from motion

Motion from form

Form from motion

Motion from form

1. Transformational Apparent Motion

Form from motion

Motion from form


2. Trackable features: Object shape can influence how fast it is perceived to rotate

Form from motion

Motion from form


2. Trackable features: Object shape can influence how fast it is perceived to rotate

3. Emergent motion signals

Overview of talk

• Theoretical background to contour discontinuities

• The role of contour discontinuities in form/motion processing• Transformational apparent motion • Motion from feature tracking• Emergent motion signals

Overview of talk



Contour discontinuities

• What are they?

• Why are they important?

Attneave, 1954

Attneave, 1954

Attneave, 1954

Attneave, 1954

Attneave, 1954

Attneave, 1954

Biederman, 1987

Biederman, 1987

Biederman, 1987

Curvature detectors

Curvature detectors

(

Wolfe, Yee & Friedman-Hill (1992).

Curvature detectors

( (

(

((

(

(

(

(

(

((

(

(

((

(( (

((

((

(

(

(

(

(

(

(

( (((

(

( (


Curvature detectors

( (

(

((

(

(

(

(

(

((

(

(

((

(( (

((

((

(

(

(

(

(

(

(

( (((

(

( (

There exist filters tuned to curvature


Curvature as a cue to 3D shape

Tse, 2002, Psych. Review

Theory: The visual system locatespoints of contour curvature discontinuityin order to generate shape-from-contours

Tse (2002, Psych. Review)

Caveat: much more is going on in shape formation









a

b

c

ba

+

=

bump ellipse

ad

d

d = 1/2(b+c)

a)

c)

b)

a

c

b

d)bumps among ellipses ellipses among bumps

Curvature discontinuities pop out

Kristjansson & Tse, 2001, P&P













There exist filters tuned to curvature discontinuities

Main theoretical point

• Contour discontinuities include:

*corners

*junctions

*terminators

*deep concavities and convexities

• These are highly informative when constructing 3D shape and 3D motion

Overview of talk


• The role of contour discontinuities in form/motion processing

• Transformational apparent motion

• Motion from feature tracking

• Emergent motion signals

Overview of talk






Overview of talk






Why did you brain match like this?

And not like this?

Does form influence matching?




?

?

• Form did not seem to play much if any role in matching. People assumed that motion-energy and the nearest neighbor principle dominated matching.

• Form did not seem to play much if any role in matching. People assumed that motion-energy and the nearest neighbor principle dominated matching.

• Tse, Nakayama, and Cavanagh argued that this was an artifact of using translational apparent motion as a probe.

Frame 1 PerceptFrame 2

Transformational Apparent Motion

Translational Apparent Motion

Transformational apparent motion

Tse, Nakayama, and Cavanagh, 1996, 1998














Tse and Logothetis, 2002


Tse and Logothetis, 2002

High-level vs Low-level motion processing

Low-level: Motion energy dependent,

nearest neighbor matching

High-level vs Low-level motion processing

Low-level: Motion energy dependent,

nearest neighbor matching

High-level: Matching takes place on the basis 3D form relationships. High-level motion paths are determined following a stage of segmentation, grouping, and 3D analysis that is particularly sensitive to contour relationships and occlusion among attended figures.

Mri data specs• GE 1.5T signa

• 1-shot T2*-weighted gradient-recalled EPI

• block design, quasi-random order

• 20 second epochs, 8 vols

• TR=2.5s, 25 slices, FA=90degrees

• voxel size: 3.75x3.75x5mm

• EPI FOV 240x240x256mm

• Data analysis BV4.9, matlab

• Stimulus back-projected, FOV 22x16 degrees

Stimuli

T C C CT T

T C CT TC

Stimuli

T C C CT T

T C CT TC

Task: press button when fixation point transientlydisappears for 17ms.

• See TAM demo

Back Right

Left Bottom

right left

rearbottom

n=19

Back Right

Left Bottom

right left

rearbottom

hMT+

n=19

Back Right

Left Bottom

right left

rearbottom

hMT+ LOC

n=19

**

**

*

*

*

Average Bilateral BOLD Response by Area (n=10; MT/LOC n=7)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

V1 V2d V2v V3d V3v V3A/B V4v MT-LOC

LOC-MT

% B

OL

D S

ign

al C

ha

ng

e

TAM Control

*

*

*

* **

*

Average Bilateral Beta Weight by Area

0

0.5

1

1.5

2

2.5

V1 V2d V2v V3d V3v V3a V4v MT-LOC

LOC-MT

Nor

mal

ized

Bet

a W

eigh

t

TAM Control

*

* **

*

*

Ventral pathway Dorsal pathway

V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d


V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d

The key idea here

• Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion.

• Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.

The key idea here



The key idea here



• Any model of motion processing that fails to explicitly incorporate a stage of global contour-based form analysis will be incomplete.

• Any model of motion processing that fails to explicitly incorporate a stage of global contour-based form analysis will be incomplete.

• For example, high-level motion cannot just be a motion-energy detector that receives input from a salience map. There must be an explicit stage of form analysis.

Overview of talk






Overview of talk






Velocity Space

The infinite number of motions that lead to the same

component vector all lie on a “constraint line” in vector

space

Percepts of Translational Motion

• The examination of Plaids – two oriented, overlapped, independently

moving sinewave gratings will appear to move as a plaid

– The direction and speed of motion can be predicted by various component-motion integration models

Integration

Example Predicted by

IOC or VS

Example Predicted by IOC

and not VS

Trackable Feature Solution to Aperture Problem

• Eyetracking and Neurophysiological Evidence suggests that certain form features (Endpoint Terminators) can be used by the visual system to disambiguate percepts of motion independent of component motion (Area MT: Pak and Born (2003)) (V1: Pak and Livingston (2003))

Components generated along a curved contour

IOC succeeds for Translational Motion





IOC Fails for Rotation






IOC for translating ellipse

IOC for rotating ellipse

• See ellipse demo

Rigidity = Preservation of Shape

Regions of High Curvature are Critical to the perception

of shape

Component Motion will always be non-rigid

Although Component Motion predicts variable speeds

Some form of form analysis must occur for stimulus to remain rigid, presumably

regions of high curvature are critical for rigidity

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Paradox!

Experiment 1

Experiment 2

Experiment 3

Experiment 4

vs.

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Determined equiluminant green for each subject

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Experiment 1

Experiment 2

Experiment 3

Experiment 4

• Now pit equiluminant ellipse against luminance-defined ellipse

vs.

• Equiluminant ellipses appear to rotate more slowly than luminance-defined ellipses

• Equiluminant ellipses appear to rotate more slowly than luminance-defined ellipses

• There is an interaction: as the ellipse gets thinner, the perceived angular velocity of an equiluminant ellipse becomes closer to that of a luminance-defined ellipse.

vs.

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Experiment 1

Experiment 2

Experiment 3

Experiment 4

Summary

• Global form-based processing of trackable features such as corners and regions of relatively high positive curvature serve as a motion cue in high-level motion processing.

• Perceived motion magnitude is a function of both low- and high-level inputs.

Question

• Which areas of the brain vary parametrically with contour curvature discontinuity points in motion?

Question

• Which areas of the brain vary parametrically with contour curvature discontinuity points in motion?

• All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.




1 2 3 4 5

• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.





All stimuli rotated at the same speed.

• See Bumps stimuli

Stimuli

1 5 23 4

2 5 134

1 2 534

Stimuli

1 5 23 4

2 5 134

1 2 534

In order to guarantee that they were maintaining fixation, observers carried out a simple task at the fixation point that was not directly related to issues of curvature processing (press button when fixation point blinks).

• Hold area constant

• Hold max velocity constant

• Hold perceived angular

velocity constant




velocity constant




velocity constant

Summary

• Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B.

• V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.

Summary



Summary




V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d


V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d

TAM results


V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d

Bumps results


V1

V2 thick stripe

V3A

MT

V2 thin stripe

V4v

MSTLOC

V3v V3d

Bumps results

Overview of talk



hMT

+

V2 V4

V3A

LOC

Ventral form system

: 3D surfaces

Dorsal form system

: Trackablefeatures

Key idea: Form determines motion

• There exist two interacting form analyses

• Ventral (V4v etc) 3D surfaces, volumes

• Dorsal (V3A etc)3D trajectories

• Both rely on contour discontinuities

Overview of talk



Conclusions

Conclusions

• Perceived motion direction follows global form analysis of contour relationships

Conclusions


• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.

Conclusions



• Perceived motion speed follows form analysis of trackable features (contour discontinuities)

Conclusions




• This form analysis occurs in V3A/B

Conclusions





• CDs play a key role in parsing and as TFs.

Conclusions





• CDs play a key role in parsing and as TFs.

• Motion perception based on emergent motion

Thank you

The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh.

Documents

form slide

pp slide

review slide

hsieh slide

d motion slide

d shape slide

curvature discontinuities

form based motion processing