Ter Haar Romeny, TU/e Mathematical Models of Contextual Operators Eindhoven University of Technology Department of Biomedical Engineering Markus van Almsick,

ter Haar Romeny, TU/e

Mathematical Models

ofContextual Operators

Eindhoven University of Technology

Department of Biomedical Engineering

Markus van Almsick, Remco Duits, Erik Franken

Bart ter Haar Romeny


Context: the Idea

What a local filter sees:What a context filter sees:


Perceptual grouping (Gestalt) from orientations: robust detection

Gestalt laws


IntroductionProblem: segmentation of curves, contours, surfaces,

etc.

Methods can be distinguished by (spatial) ‘locality’

Local Global

Pixelwise Local filters

/derivativesContext operators Active contours,

ASM, etc.

E.g. threshold on pixel values

Pro: computationally efficientCon: only applicable on very ‘clean’ images

E.g. Gaussian derivatives+threshold/local max

Pro: pretty efficientCon: sensitive to noise or inconsistent data if features “live” at low scale in scale-space

Optimization of global cost functional based on smoothness constraints (+ shape/texture knowledge)

Pro: effective and stable on specific class of objectsCon: needs initial estimate, (prior shape knowledge)

Operators that take a “larger context” into account, by enhancing local features using context model.

Pro: noise-robust, limited amount of prior knowledgeCon: computational expensive


Context: the Empirics

Angular specifity in the striate cortex: voltage sensitive dye recording of cortical colums. Similar orientations are connected (even over great distances) – “probability voting”.

“Orientation selectivity and the arrangement of

horizontal connections in tree shrew striate cortex”

W.H.Bosking, Y Zhang, Y.Schofield, D.Fitzpatrick

(1997) J. Neuroscience 17:2112-2127


Goal: Extracting Edges, Lines and Surfacesfrom noisy, low dose, or fastly acquired medical

images


Overview

• Invertible Orientation Bundle

TransformationThe output of the oriented filters spans a new transformed

space, like the Fourier transform. An inverse transform can be

found!

• Tensor Voting


Template Matching

imagekernelresponse

Classical filters


G-Convolution

symmetry transformation g

g dependence

Classical filters


Linear Convolution Filter

translation by b

Classical filters


Wavelet Transform

dilation a translation b

Classical filters


Orientation Bundle Transform

rotation α translation b

New filter family


Orientation Bundle Transform

QuickTime™ and aAnimation decompressor

are needed to see this picture.


Measures

L2 inner product by Euclidean measure

L2 inner product by Haar measure

image response


Inverse Transformation

Kernel Constraint


Gaussian Orientation Bundle

Harmonic amplitudes are constructed from the local Gaussian derivative jet

0;)(),(

,)(),(

2

2

neaz

eazzz

nnn

zzn

nn


RemcoDuits:

InvertibleOrientationWaveletTransform[Siam2004]

Best paperaward atPRIA 2004


Strong non-linear filtering in orientation spacegives a much better detection of very dim lines in noise

{x,y} OS

OS OS6

OS6 {x,y}


Finding the very thin Adamkiewiczvessel in aorta reconstructive surgery:Not reconnecting may give spinal lesion.

3D waveletfor invertibleorientationtransform

Noisy original Denoised vessel


Orientation Bundle Transform• invertible

• isometric

• variety of admissible kernels

This gives a new ‘space’ for geometricreasoning


Context: Autocorrelation of Luminosity


Autocorrelation of Edges


Autocorrelation of Lines


Autocorrelation of Lines


Tensor voting

Voting kernel


Steerable Tensor Voting


Context filters for dim and broken contour detection

Ultrasound kidney Context-enhanced

Contour extraction

Local

Contour extraction


Vessel detectionfor ComputerAided Diagnosisin mammography

E. Franken, M. van Almsick


Application: Cardiac ElectrophysiologyTreatment of heart rhythm disorders

1. Insertion of EP catheters

2. Recording of intracardiac electrograms

3. Ablation of problematic spot, or blocking undesired conduction path

Erik Franken, 2006


Example - input

Source image

Local ridgeness

Erik Franken, 2006


Example - result

Context enhanced ridgeness

*

*

*

*

*

+

+

+

+

U2(x,y)=|U2|

Erik Franken, 2006


Repeated tensor voting

Tensor voting thinning tensor voting

Result after first step Result after second step

Erik Franken, 2006


Fluoroscopyat 1/50 of theregular dose







Extracted most salient paths

Extraction of paths

Extracted catheter tips

Erik Franken, 2006


Extension of catheter tips

Selection of the best extension candidate for each

tip.

Result:

Erik Franken, 2006


Evaluation of extraction results

20

40

60

80

100%

Low noise High noise

TV

No TV

TV

No TV

%entire

%tip

Erik Franken, 2006


Sarcomers – bands of overlappingactine – myosine molecules inmuscle fibres

Orientation score - nonlinar diffusion

Ter Haar Romeny, TU/e Mathematical Models of Contextual Operators Eindhoven University of Technology Department of Biomedical Engineering Markus van Almsick,

Documents

voting kernel slide

autocorrelation of edges

van almsick slide

medical images slide

regular dose slide

computational expensive

geometric reasoning

steerable tensor voting