Front End Vision: A Multiscale Geometry Engine Scale-Space Theory in Computer Vision versus Front-End Biological Vision Bart ter Haar Romeny 1 , PhD & Luc M .J .Florack 2 , PhD Utrecht University : Image Sciences Institute 1 , Dept.of Computer Science 2 3584 CX Utrecht, the Netherlands B . terHaarRomeny isi . uu . nl & florack cs . uu . nl Introduction The front end visual system belongs to the best studied brain areas. Scale-space theory, as pioneered by Iijima in Japan [11, 41, 43] and Koenderink [12] has been heavily inspired by the important derivation of the Gaussian kernel and its derivatives as regularized differential operators, and the linear diffusion equation as its generating PDE. The view visual system as a ’geometry’ engine is the inspiration of the current work, and simultaneously, the presented examples of applications of (differential) geometric operations may inspire the thinking of the visual system as a geometry engine. Scale-space theory has developed into a serious field [34, 20]. Several comprehensive overview texts have been published in the field [15, 10, 40]. The introduction of a geometry driven conduction term in the diffu- sion equation, making it locally adaptive to differential geometric properties (edge strength, curvature, orienta- tion) by Perona and Malik in the early nineties triggered a wealth of nonlinear PDE developments, which attracted the attention of the mathematical community. So far, however, this robust mathematical framework has seen impact on the computer vision community, but there is still a gap between the more physiologically, psycologically and psychophysically oriented researchers in the vision community. One reason may be the nontrivial mathematics involved, such as group invariance, differential geometry and tensor analysis. The last couple of years symbolic computer algebra packages, such as Mathematica, Maple and Matlab, have developed into a very user friendly and high level prototyping environment. Especially Mathematica combines the advantages of symbolic manipulation and processing with an advanced front-end text proces- sor. This paper has been completely written in Mathematica version 4 as a notebook. The advantage is that this paper can be read as an interactive paper: the high level code of any function is directly visible, and can be operated directly, as well as modified or templated for own use. Students can now use the exact code rather then pseudocode. With these high level programming tools most programs can be expressed in very few lines, so it keeps the reader at a highly intuitive but practical level. Mathematica notebooks are portable, and run on any system equivalently. Previous speed limitations are now well overcome.
34
Embed
Front End Vision: A Multiscale Geometry Enginefaculty.petra.ac.id/resmana/private/pcd/Image-Processing...Front End Vision: A Multiscale Geometry Engine Scale-Space Theory in Computer
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Front End Vision: A Multiscale Geometry Engine
Scale-Space Theory in Computer Visionversus Front-End Biological Vision
Bart ter Haar Romeny1, PhD & Luc M.J.Florack2, PhD
The front end visual system belongs to the best studied brain areas. Scale-space theory, as pioneered by
Iijima in Japan [11, 41, 43] and Koenderink [12] has been heavily inspired by the important derivation of the
Gaussian kernel and its derivatives as regularized differential operators, and the linear diffusion equation as
its generating PDE. The view visual system as a ’geometry’ engine is the inspiration of the current work, and
simultaneously, the presented examples of applications of (differential) geometric operations may inspire the
thinking of the visual system as a geometry engine.
Scale-space theory has developed into a serious field [34, 20]. Several comprehensive overview texts have
been published in the field [15, 10, 40]. The introduction of a geometry driven conduction term in the diffu-
sion equation, making it locally adaptive to differential geometric properties (edge strength, curvature, orienta-
tion) by Perona and Malik in the early nineties triggered a wealth of nonlinear PDE developments, which
attracted the attention of the mathematical community.
So far, however, this robust mathematical framework has seen impact on the computer vision community,but there is still a gap between the more physiologically, psycologically and psychophysically oriented
researchers in the vision community. One reason may be the nontrivial mathematics involved, such as group
invariance, differential geometry and tensor analysis.
The last couple of years symbolic computer algebra packages, such as Mathematica, Maple and Matlab,
have developed into a very user friendly and high level prototyping environment. Especially Mathematica
combines the advantages of symbolic manipulation and processing with an advanced front-end text proces-
sor. This paper has been completely written in Mathematica version 4 as a notebook. The advantage is that
this paper can be read as an interactive paper: the high level code of any function is directly visible, and can
be operated directly, as well as modified or templated for own use. Students can now use the exact code
rather then pseudocode. With these high level programming tools most programs can be expressed in very
few lines, so it keeps the reader at a highly intuitive but practical level. Mathematica notebooks are portable,
and run on any system equivalently. Previous speed limitations are now well overcome.
The main focus of the paper is twofold: to provide a rehearsal of the derivation of the Gaussian kernel and its
derivatives as an essential class of front-end vision aperture functions, and to provide a practical tutorial for
a broad audience to be able to do geometric reasoning with robust multiscale differential operators on dis-
crete images. This may break ground for the view of the front-end visual system as a geometry-engine, or
'inference machine', rather then do a spatial frequency analysis.
This paper can only focus on a small area, the differential geometry and its features. Much research is cur-
rently underway. An important area is especially the deep structure of images, where the relations between
scales are studied.
Initialization
We first initialize Mathematica with a path to the image directory, load some Graphics packages, set some
often used options for some plotfunctions, turn off the spellingchecker and opimize for speed and memory.
The axiom of isotropy translates into the fact that we now only have to consider the length of our spatial
vector: { µ x, ¶ y} = ·¹¸º º » ||¼ ||.
The axiom of scale-invariance is the core of the reasoning: when we observe (blur) an observed image again,
we get an image which is blurred with the same but wider kernel:½¿¾�ÀÂÁ 1 ÃÅıÆ�ÇÉÈ 2 ÊÌËÎͱÏ�ÐÉÑ 1 ÒÔÓÉÕ 2 Ö .A general solution of this equation is: ׿Ø�ÙÉÚÂÛÌÜ exp ÝÞÝ�ßÔàÉáÉâ p ã . We must raise the argument here to the power
of p because we are dealing with the dimensionless parameter äÉå .
The dimensions are independent, thus separable: æçæéèëêì ìÔí$îçîEïñðëò1 óÂô eõ 1 öø÷�ù 1 úÂû eü 2 ý ... where eþ i are the basis
unit coordinate vectors.
The magnitude of ÿçÿ�� �� ������� is calculated by means of Pythagoras from the projections along e i, so we add the
squares, i.e. p 2. We further demand the solution to be real, so � 2 is real. We notice that when we open the
aperture fully, we blur everything out, so lim �� �� 0 ����������� 0 . This means that � 2 must be negative. We
choose � 2 � � 1! ! ! !2
and finally get the answer: "�#�$&%' ' , (�)+* exp,.- 1/ / / /2 0 2 1 2 2 , which is in the spatial domain:
This is the Gaussian kernel, which is the Green's function of the linear, isotropic diffusion equationI 2LJ J J J J J J J JKx2 LNM 2LO O O O O O O O O OP
y2 Q Lxx R Lyy SUT LV V V V V V VW s, where s X 2 Y 2 is the variance. Note that the derivative to scale is here the
derivative to Z 2, which also immediately follows from a considerations of the dimensionality of the equation.
4 Romeny-c.nb
All partial derivatives of the Gaussian kernel are solutions too of the diffusion equation.
So the first important result is that we have found the Gaussian kernel and all of its partial derivatives as the
unique kernel for a front-end visual system that satisfies the constraints "no preference for location, scale and
orientation" and linearity. We have found a one-parameter family of kernels, where the scale [ is the free
parameter. This is a general feature of the biological visual system: the exploitation of ensembles of aperture
functions, which are mathematically modeled by families of kernels for a free parameter, e.g. for all scales,
derivative order, orientation, stereo disparity, motion velocity etc.
The Gaussian kernel is the unique kernel that generates no spurious resolution (e.g. the squares so familiar
with zooming in on pixels). It is the physical point operator, the Gaussian derivatives are the physical deriva-
tive operators.
Gaussian partial derivative kernels
Here are the receptive field sensitivity structures of some members of the Gaussian derivative family:
Upper left: the Gaussian kernel as the zero-th order operator; upper right: Æ GÇ Ç Ç Ç Ç Ç Ç ÇÈx
; lower left: É 2GÊ Ê Ê Ê Ê Ê Ê ÊÊ Ê Ê ÊËx Ì y ; lower right:
retinal and LGN center-surround receptive fields are well modeled by the (positive resp. negative) LaplaceanÍ 2GÎ Î Î Î Î Î Î Î Î ÎÏx2 ÐÒÑ 2GÓ Ó Ó Ó Ó Ó Ó Ó Ó ÓÔ
y2 of the Gaussian kernel.
The receptive fields in the primary visual cortex closely resemble Gaussian derivatives, as was first noticed
by Young [Young 1984, 1986] and Koenderink [Koenderink 1984], and they may accomplish a double
simultaneous task: observation and differentiation. These RF’s come at a wide range of sizes, and at all
orientations.
Below two examples are given of the measured receptive field sensitivity profile of a cortical simple cell (left)
and a Lateral Geniculate Nucleus (LGN) center-surround cell, as measured by DeAngelis, Ohzawa and
Freeman [4], [http://totoro.berkeley.edu/].
Left: cortical simple cell, well modeled by a first order Gaussian derivative kernel. Right: center-surround
LGN cell, well modeled by the Laplacean of a Gaussian. From [4].
Through the center-surround structure at the very first level of measurement on the retina the Laplacean of
the input image can be seen to be taken. The linear diffusion equation states that this Laplacean is equal to
the first derivative to scale: Lxx Õ Lyy ÖØ× LÙ Ù Ù Ù Ù Ù ÙÚ s. One conjecture for its presence at this level could be that the
visual system actually might measure Û LÜ Ü Ü Ü Ü Ü ÜÝs, i.e. the slight change in signal Þ L when the aperture is changed
with ß s: at homogeneous areas there is no output, at highly textured areas there is much output. Integrating
both sides of à L áãâ Lxx ä Lyy åçæ s over all scales gives the measured intensity in a robust fashion.
Derivatives of sampled, i.e. observed data
The derivative of the observed data L0 è x, yé{ê Gë x, y; ì�í is given by îï ï ï ï ï ïðx ñ L0 ò x, yó{ô Gõ x, y; ö�÷ùø , which can
be written as L0 ú x, yûýüÿþ� � � � � ��x
G�x, y; ��� . The commutation of the convolution and the derivative operators is
possible because of their linearity, which is easily shown in the Fourier domain. From this we can see the
following important results:� Differentiation and observation can be done in a single step: convolution with a Gaussian derivative kernel.� Differentiation is now done by integration, i.e. by the convolution integral.� The Gaussian kernel is the physical analogon of a mathematical point, the Gaussian derivative kernels are
the physical analogons of the mathematical differential operators. Equivalence is reached for the limit when
6 Romeny-c.nb
the scale of the Gaussian goes to zero: lim� 0 G� x; �������� x� , where ��� x� is the Dirac delta function, and
x # $ x.% Any differention blurs the data somewhat, with the amount of the scale of the differential operator. There
is no way out this increase of the inner scale, we can only try to minimize the effect.
The Gaussian kernel has by definition a strong regularizing effect. It was shown by Schwartz [39] that
differentiation of distributions of data (such as sampled data) has to be accomplished by convolution with a
smooth testfunction. It is important to realize that the process of observation is the regularizer. Recently some
interesting papers have shown the complete equivalence of Gaussian scale space regularization with a num-
ber of other methods for regularization [19, 38].
There have been published many formulations to derive the front-end aperture function as the Gaussian
kernel and its derivatives. For an overview see Weickert [41] and Lindeberg[17].
Gabor kernels
The derivation given below required first principles be plugged in that essentially stated "we know knothing"
(at this stage of the observation). Of course, we can relax these principles, and introduce some knowledge.
When we want to derive a set of apertures tuned to a specific spatial frequency k& in the image, we add thisphysical quantity to the matrix of the dimensionality analysis:'()))))))
Following the exactly similar line of reasoning, we end up from this new set of constraints with a new family
of kernels, the Gabor family of receptive fields, with are given by a sinusoidal function (at the specified
spatial frequency) under a Gaussian window:
gabor @ x_ , A _ B : C Sin D x E 1F F F F F F F F F F F F F F F FF F F F F FGIH H H H H H H H H H H H H H2 JLK 2
Exp MON x2P P P P P P P PP P P2 Q 2 R ;
Note the similarity between Gabor and Gaussian derivative kernels. They can be made to look very similar by
an appropriate choice of parameters:
Romeny-c.nb 7
gauss S x_ , T _ U : V 1W W W W W W W W W W W W W W W WW W W W W WXZY Y Y Y Y Y Y Y Y Y Y Y Y Y2 []\ 2
Exp ^O_ x2` ` ` ` ` ` ` `` ` `2 a 2 b ;
p1 c Plot dOe gabor f x, 1 g , h x, i 4, 4 j ,PlotStyle k Dashing lnm 0.02 , 0.02 oqp , DisplayFunction r Identity s ;
p2 t Plot u Evaluate v D w gauss x x, 1 y , x z�z , { x, | 4, 4 } , DisplayFunction ~ Identity � ;Show �n� p1, p2 � , DisplayFunction � $DisplayFunction � ;
-4 -2 2 4
-0.2
-0.1
0.1
0.2
The essential difference is that Gabor functions have an infinite number of zero crossings, the Gaussian
derivatives as many as the order of differentiation. By relaxing or modifying other constraints, we might find
other families of kernels. We conclude this section by the realization that the front-end visual system at the
retinal level has a task to be uncommitted, no feedback from higher levels is at stake, so the Gaussian kernel
seems a good candidate to start exploring with at this level. The extensive feedback loops from the primary
visual cortex to LGN may give rise to ’geometry-driven diffusion’ [30], nonlinear scale-space theory, where
the early differential geometric measurements through e.g. the simple cells may modify the kernels at other
levels. Nonlinear scale-space theory will be extensively treated in a forthcoming interactive paper.
Differential geometry and invariance
It is essential to work with descriptions that are independent of the choice of coordinates. This was Ein-
stein's impetus in his development of the general theory of relativity. This means, that when we apply a
transformation on our coordinates, we like our local image properties to be independent of this transforma-
tion. E.g. if we rotate our {x,y} coordinate frame, we do not want locale measures as edge strength or curva-
ture to be changed. Coordinate transformations can be divided in groups. E.g. all coordinate transformations
that leave the axes of the coordinates perpendicular (e.g. rotations, translations, mirroring and scaling) form
the group of the orthogonal transformations. Another group is the group of the affine transformations, where
the new coordinates {x',y'} are acquired through a linear transformation applied to the original coordinates
{x,y}, with a, b, c and d constants:� x’y’ � = � a b
c d ��� x y �Affine transformations occur when we view objects obliquely at a relatively large distance. At shorter dis-
tances such views are described with perspective transformations.
8 Romeny-c.nb
Entities that do not change under a group of coordinate transformations are called invariants under that
particular group. The only geometrical entities that make physically sense are invariants. In the words of
Hermann Weyl: "any invariant has a specific meaning", and as such they are widely studied in computer
vision theories.
In this paper we only study orthogonal and affine invariants, as they form an important basic group and are
often encountered in computer vision.
Multiscale derivatives: implementations
In order to get some feeling of the interactive use of Mathematica, we start in this section with three implemen-
tations of convolution with a Gaussian derivative kernel (in 2D): implementation in the Fourier domain, in the
spatial domain with a 2D kernel, and in the spatial domain exploiting the separability property through two
1D kernel convolutions. Blurring, i.e. convolution with the plain Gaussian kernel, is done through convolu-
tion with the zero order Gaussian derivative.
The function gDf [ i m, nx, ny, � ] implements the convolution of the image with the Gaussian derivative
for 2D data in the Fourier domain. This is an exact function, no approximations other then the finite periodic
window in both domains. We explicitly give the code of the functions here, so you see how it is implemented,
the reader may make modifications as required. For Mathematica novices: all information on (capitalized)
internal functions is on board in the Help Browser (highlight+key F1).
Variables: i m = 2D image (as a list structure)
nx, ny = order of differentiation to x resp. y� = scale of the kernel, in pixels
The error is defined as the amount of the energy (the square) of the kernel that is ’leaking’ relative to the total
area under the curve (note the integration ranges):
error ¸ n_, ¹ _ º¼» 100
½¿¾ À�ÁI Â;à 2 n fftgauss Ä�Å , Æ;Ç 2 È?ÉÊ Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê ÊÊ Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê Ê ÊÊ Ê Ê Ê Ê Ê Ê ÊÊ Ê Ê Ê Ê ÊË
0Ì�Í I Î;Ï 2 n fftgauss Ð�Ñ , Ò;Ó 2 Ô?Õ100 Ö¡Ö 1 × 2 n Ø Gamma Ù 1Ú(Ú Ú Ú
2 Û n Ü�Ý 2 Gamma Þ 3ß(ß ß ß2 à n áãâdä 1 å 2 n æ 2 Gamma ç 1è(è è è2 é n, ê 2 ë 2 ì¡íî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(î(î(î(î(î(î(î(î(îî(î(î(î(î(î(î(îî(î(î(î î(îï
1 ð 2 n ñ 2 Gamma ò 1ó(ó ó ó2 ô n õ
16 Romeny-c.nb
We plot this Gammafunction for scales between ö�÷ 0.2 ø 2 and order of differentiation from 1 to 10, and we
insert the 5% error line in it (we have to lower the plot somewhat to make the line visible):
Block ùqú $DisplayFunction û Identity ü ,p1 ý Plot3D þ error ÿ n, � ��� 6, ��� , .2 , 2 � , � n, 1, 10 , PlotRange All ,
val 6 Apply 7 error , First 8 pts 9#9 ; Line : Map ; Append < #, val = &, pts >?>#@BAC>#> ;Show D p1, c3d E ;
0.51
1.52F
24
6810n
0
25
50
75
error %
0.51
1.52G
24
6810n
The lesson from this section is that we should never make the scale of the operator, the Gaussian kernel, too
small. The lower limit is indicated in the graph above. A similar reasoning can be set up for the outer scale,
when the aliasing occurs in the spatial domain.
Natural coordinates
The intensity of images and invariant features at larger scale decreases fast. This is due to the non-scaleinvari-
ant use of the differential operators. For, if we consider the transformation xH H H H HIKJ xL , then xM is dimensionless.
At every scale now distances are measured in a distance yardstick with is scaled with the scale itself, i.e.
scale-invariant. The dimensionless coordinate is termed the natural coordinate. This implies that the deriva-
tive operator in natural coordinates has a scaling factor: N nO O O O O O O O OPxQ n RTS n U nV V V V V V V V VW
xn .
Here we generate a scale-space of the intensity gradient. To study the absolute intensities, we plot every
image with the same intensity plotrange of {0,40}:
Romeny-c.nb 17
im X Import Y " mr128 . gif " Z)[#[ 1, 1 \#\ ;p1 ] Table ^ grad _,`ba a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a agD c im , 1, 0, dfe 2 g gD h im , 0, 1, ifj 2 ;k
ListDensityPlot l grad , PlotRange m#npo 0, 40 q , DisplayFunction r Identity s ,ListDensityPlot tvu grad , PlotRange w#xby 0, 40 z , DisplayFunction { Identity |#}
Show C�D p1, frame E , DisplayFunction F $DisplayFunction ,
Frame G False , ImageSize HJI 150, 150 K�L ;
v
w
The derivatives to v and w are by definition features that are invariant under orthogonal transformations, i.e.
rotation and translation. To apply these gauge derivative operators on images, we have to convert to the
Cartesian M x, y N domain. The derivatives to v and w are defined as:Ov PRQ Ly S x T Lx U yV V V V V V V V V V V V V V V VV V V V V V V VV V V V V V VWYX X X X X X X X X X X X X X X X X X X X
w c Lx d x e Ly f yg g g g g g g g g g g g g g g gg g g g g g g gg g g ghji i i i i i i i i i i i i i i i i i i iLx2 k Ly2 l Li m ij n jo o o o o o o oo o o o o o o o opaq q q q q q q q q q q
Li Li.
We can alternatively see the derivatives to r v, ws as rotated over an angle t , with rotation matrixu cos vxwzy sin {}|$~� sin �}�(� cos �}�$������� Lx Ly� Ly Lx � . The second formulation uses tensor notation, where the index i or j stands for the range of dimensions. So
Li ��� Lx, Ly � in 2D and Li ��� Lx, Ly, Lz � in 3D. Likewise � j is the nabla operator ���� � � � � ��x
, �� � � � � � ��y � . The constant
tensors � ij and � ij are the symmetric Kronecker tensor 1 00 1 ¡ and the antisymmetric Levi-Civita tensor¢ 0 £ 1
1 0 ¤ respectively (in 2D). With this notation, we see that the derivative operator ¥ w is defined as the
derivative operator ¬ v is defined as the derivative operator j rotated in the perpendicular direction (through®ij ) of the unit length gradient vector Li¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯°a± ± ± ± ± ± ± ± ± ± ±
Li Li. So we encounter 3 types of notation for our geometrical exer-
20 Romeny-c.nb
cises: Cartesian coordinate notation ² x, y ³ , tensor notation Li Li, and gauge coordinate notation ´ v, wµ . They
are essentially equivalent. In this paper we will elaborate mainly on gauge coordinates.
The definitions above are easily accomplished in Mathematica:
¶2 · IdentityMatrix ¸ 2 ¹ºaº1, 0 » , ¼ 0, 1 ½a½
¾ 2 ¿ Table À Signature Á� i , j Ã�Ä , Å j , 2 Æ , Ç i , 2 È�ÉÊaÊ0, Ë 1 Ì , Í 1, 0 ÎaÎ
dv Ô 1Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ Õ ÕÕ Õ Õ Õ Õ Õ Õ Õ Õjacobean Ö Lx , Ly × . Ø 2. Ù D Ú #, x Û , D Ü #, y Ý�Þ &
ßLx, Ly à . á 2. âäã x#1, å y#1 æçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèçèççèçèçèçèçèçèçèçèçèçèçèçèçèçèçèççèçèçèçèçèçèçèççèçèçèçèçèçèçèççèç ç
jacobean&
dw é 1ê ê ê ê ê ê ê ê ê ê ê ê ê ê ê êê ê ê ê ê ê ê ê êjacobean ë Lx , Ly ì . í 2. î D ï #, x ð , D ñ #, y ò�ó &
Lww * dw + dw , L - x, y .�.�.0/ . 1 Lx 2�3 x L 4 x, y 5 , Ly 6�7 y L 8 x, y 9�:�;�; Simplify
<L = 0,1 >#? x, y @ 2 L A 0,2 B#C x, y D E 2 L F 0,1 G#H x, y I L J 1,0 K#L x, y M L N 1,1 O#P x, y Q RL S 1,0 T#U x, y V 2 L W 2,0 X#Y x, y Z\[^]`_ L a 0,1 b#c x, y d 2 e L f 1,0 g#h x, y i 2 j
Lvv k dv l dv m L n x, y opo�o0q . r Lx s�t x L u x, y v , Ly wyx y L z x, y {�|�}�} Simplify
We also recognize Lvv in the ’fundamental’ equation of Alvarez et al. [2], a nonlinear geometry driven diffu-
sion equation: , L-.-.-/- - -0s 1 Lvv.
Romeny-c.nb 23
Isophote curvature in gauge coordinates
Isophote curvature 2 is defined as the change w’’ of the tangent vector w’ in the gauge coordinate system.
When we differentiate the definition of the isophote (L = Constant) to v, we get:
D 3 L 4 v , w 5 v 6�687�7 Constant , v 9w :�; v < L = 0,1 >@? v, w A v BCBED L F 1,0 G@H v, w I v JKJMLCL 0
We know that Lv N 0 by definition of the gauge coordinates, so w’ = 0, and the curvature O = w’’ is found by
differentiating the isophote equation again:
PRQ w’’ S v T8U . Solve V D W L X v , w Y v Z&Z\[�[ Constant , ] v , 2 ^�_8` . w’ a v b%c�d 0, w’ ’ e v f�fgih L j 2,0 k@l v, w m v nKnopopopopopopopopopopopopopopopoopopopopopopopopopopopopopopopoopopopo opo o
L q 0,1 r@s v, w t v uKuwvSo xMy{z Lvv| | | | | | | | |
Lw. QED.
In Cartesian coordinates we recognize the well-known formula from classical textbooks:
}R~�� dv � dv � L � x, y ���&�� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �� � � � � � � �� � � � �dw � L � x, y ��� � . � Lx �R� x L � x, y � , Ly ��� y L � x, y �&�\��� Simplify
When we study the curvature of the isophotes in the middle of the image, at the location of the T-junction, we
see the isophote ’sweep’ from highly curved to almost straight for decreasing intensity. So the geometric
reasoning is the "the isophote curvature changes a lot when we traverse the image in the w direction". It
seems to make sense to study ���� � � � � � � w
. We saw before that the isophote curvature ! is defined as "$#&% Lvv' ' ' ' ' ' ' ' 'Lw
. So
the Cartesian expression for the T-junction detector becomes
(*),+ dv - dv . L / x, y 0�0102 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 22 2 2 2 2 2 2 22 2 2 2 2dw 3 L 4 x, y 5�5
6. 7 Lx 8*9 x L : x, y ; , Ly <>= y L ? x, y @1ACB�B Simplify ;
tjunction D Simplify E dw FHGJICK . L Lx M*N x L O x, y P , Ly QSR y L T x, y U�V�U ;% W . Derivative X n_, m_Y[Z L \[] x, y ^`_ " L" acb Table d " x" , e n f1gihkj Table l " y" , m mn�o
It looks like an impossibly complicated polynomial in fourth order derivative images, and it is. Through the
use of Gaussian derivative kernels each separate term can easily be calculated. We change all coefficientsinto scaled Gaussian derivatives:
discr4 6 im_ , 7 _ 8 : 9Discriminant : pol4 , x ;=< . > y ? 1, Lxxxx @ gD A im , 4, 0, BDC , Lxxxy E gD F im , 3, 1, GDH ,
Lxxyy I gD J im , 2, 2, KML , Lxyyy N gD O im , 1, 3, PMQ , Lyyyy R gD S im , 0, 4, TMUWV
Let us apply this high order function on an image of a checkerboard, and we add noise with twice the maxi-
mum image intensity to show its robustness, despite the high order derivatives:
30 Romeny-c.nb
t1 X Table Y If ZW[ x \ 50 && y ] 50 ^`_W_ba x c 50 && y d 50 e , 0, 100 fhg 200 i Random jlk ,mx, 1, 100 n , o y, 1, 100 pWq ;
t2 r Table s If tWu x v y w 100 x 0 && y y x z 0 {}|W|�~ x � y � 100 � 0 && y � x � 0 � , 0, 100 ���200 � Random ��� , � x, 1, 100 � , � y, 1, 100 �W� ;
[13], deep structure of images etc. Space constraints do not allow to elaborate on these issues in this paper.
This paper is an exerpt from a forthcoming book [35], where many of the issues above are treated in an
interactive way.
This paper has been written as a notebook in Mathematica 4.0, giving the possibility to the reader to experi-
ment with every treated subject himself. The high level of functions and speed of code and hardware now
available (this full notebook runs in 12 minutes on a Pentium II PC, 266 MHz, 128K memory, Win95) and
the easy interactive visualization possibilities makes the combination of textbook text and code a highly
tutorial toolkit on the desktop.
Romeny-c.nb 31
Acknowledgement
The authors thanks the members of the Image Sciences Institute, particularly the members of the TGV
(’Tools of Geometry in Vision’) team for their contributions and discussions. Special thanks to Max Vier-
gever, director of ISI.
References[1] P. Abbott (ed.), "Tricks of the Trade", The Mathematica Journal, Wolfram Media Inc., vol. 7, no.2, 105-127, 1998.
[2] L. Alvarez, F. Guichard, P. L. Lions, and J. M. Morel. Axiomes et equations fondamentales du traitement d’images. C. R. Acad. Sci. Paris, 315:135--138, 1992.
[4] Gregory C. DeAngelis, Izumi Ohzawa, and Ralph D. Freeman, "Receptive-field dynamics in the central visual pathways", Trends Neurosci. 18: 451-458, 1995.
[5] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, "Scale and the differential structure of images," Image and Vision Computing, vol. 10, pp. 376-388, July/August 1992.
[6] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, "Cartesian differential invariants in scale-space," Journal of Mathematical Imaging and Vision, vol. 3, pp. 327-348, November 1993.
[7] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, "Images: Regular tempered distributions," in Proceedings of the NATO Advanced Research Workshop Shape in Picture - Mathematical description of shape in greylevel images (Y.-L. O, A. Toet, H. J. A. M. Heijmans, D. H. Foster, and P. Meer, eds.), vol. 126 of NATO ASI Series F, pp. 651-660, Springer Verlag, Berlin, 1994.
[8] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, "Linear scale-space," Journal of Mathematical Imaging and Vision, vol. 4, no. 4, pp. 325-351, 1994.
[9] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, "The Gaussian scale-space paradigm and the multiscale local jet," International Journal of Computer Vision, vol. 18, pp. 61-75, April 1996.
[10] L. M. J. Florack, "Image Structure", Kluwer Academic Publishers, Dordrecht, the Netherlands, 1997.
[11] T. Iijima: "Basic Theory on Normalization of a Pattern (in Case of Typical 1D Pattern). Bulletin of Electrical Laboratory, vol. 26, pp. 368-388, 1962 (in Japanese).
[12] J.J. Koenderink: "The Structure of Images", Biological Cybernetics, vol. 50, pp. 363-370, 1984.
[14] T. Lindeberg: "Scale-space for discrete signals’’, IEEE Transactions of Pattern Analysis and Machine Intelligence, 12(3), 234--254, 1990.
[15] T. Lindeberg: "Scale-Space Theory in Computer Vision", Kluwer Academic Publishers, Dordrecht, Netherlands, 1994.
[16] T. Lindeberg: "Scale-space theory: A basic tool for analysing structures at different scales’’, J. of Applied Statistics, 21(2), Supplement on Advances in Applied Statistics: Statistics and Images: 2, pp. 224--270, 1994.
[17] T. Lindeberg, "On the axiomatic foundations of linear scale-space’’. In J. Sporring, M. Nielsen, L. Florack, and P. Johansen (eds.) Gaussian Scale-Space Theory, 75-97, Kluwer Academic Publishers, 1997.
32 Romeny-c.nb
[18] P. Morrison, "Powers of Ten: About the Relative Size of Things in the Universe",W. H. Freeman and Company, 1985.
[19] M. Nielsen, "Scale-Space Generators and Functionals’’. In J. Sporring, M. Nielsen, L. Florack, and P. Johansen (eds.) Gaussian Scale-Space Theory, pp. 99-114, Kluwer Academic Publishers, 1997.
[20] M. Nielsen, P. Johansen, O.F. Olsen, J. Weickert (Eds.), Scale-space theories in computer vision, Lecture Notes in Computer Science, Vol. 1682, Springer, Berlin, 1999.
[21] W. J. Niessen, B. M. ter Haar Romeny, L. M. J. Florack, and M. A. Viergever, "A general framework for geometry-driven evolution equations," International Journal of Computer Vision, vol. 21, no. 3, pp. 187-205, 1997.
[22] W. J. Niessen, B. M. ter Haar Romeny, L. M. J. Florack, A. H. Salden, and M. A. Viergever, "Nonlinear diffusion of scalar images using well-posed differential operators," in Proc.of Computer Vision and Pattern Recognition, (Seattle, WA), pp. 92-97, IEEE Computer Society Press, 1994.
[23] E. Radmoser, O. Scherzer, J. Weickert, "Scale-space properties of nonstationary iterative regularization methods", to appear in Journal of Visual Communication and Image Representation (Special Issue on Scale-Space Theories in Computer Vision, invited paper), 2000.
[24] B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, "Invariant third order properties of isophotes: T-junction detection," in Proc.7th Scand.Conf.on Image Analysis (P. Johansen and S. Olsen, eds.), Aalborg DK, pp. 346-353, August 1991.
[25] B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, "Scale-space: Its natural operators and differential invariants," in Information Processing in Medical Imaging (A. C. F. Colchester and D. J. Hawkes, eds.), vol. 511 of Lecture Notes in Computer Science, pp. 239-255, Springer-Verlag, Berlin, July 1991.
[26] B. M. ter Haar Romeny and L. M. J. Florack, "A multiscale geometric model of human vision," in Perception of Visual Information (W. R. Hendee and P. N. T. Wells, eds.), ch. 4, pp. 73-114, Berlin: Springer-Verlag, 1993. Second edition 1996.
[27] B. M. ter Haar Romeny, L. M. J. Florack, A. H. Salden, and M. A. Viergever, "Higher order geometrical image structure," in Proc. Information Processing in Medical Imaging ’93, Flagstaff AZ (H. Barrett, ed.), (Berlin), pp. 77-93, Springer-Verlag, 1993.
[28] B. M. ter Haar Romeny, L. M. J. Florack, A. H. Salden, and M. A. Viergever, "Higher order differential structure of images," Image and Vision Computing, vol. 12, pp. 317-325, July/August 1994.
[29] B. M. ter Haar Romeny, W. J. Niessen, J. Wilting, and L. M. J. Florack, "Differential structure of images: Accuracy of representation," in Proc. First IEEE Internat. Conf. on Image Processing, (Austin, TX), pp. 21-25, IEEE, November, 13-16 1994.
[30] B. M. ter Haar Romeny (Ed.), "Geometry Driven Diffusion in Computer Vision". Series on Computational Imaging and Vision, Kluwer Academic Publishers, Dordrecht, the Netherlands, 1994.
[31] T. Lindeberg and B. M. ter Haar Romeny, "Linear scale-space: I. basic theory, II.Early visual operations". In: Geometry-Driven Diffusion in Computer Vision (B. M. ter Haar Romeny, ed.), Computational Imaging and Vision, pp. 1-38,39-72, Dordrecht, the Netherlands: Kluwer Academic Publishers, 1994.
[32] B. M. ter Haar Romeny, "Scale-space research at Utrecht University," in Proc. 12th International Conference on Analysis and Optimization of Systems: Images, Wavelets and PDE’s (M.-O. Berger, R. Deriche, I. Herlin, J. Jaffré, and J.-M. Morel, eds.), Lecture Notes in Control and Information Sciences, vol. 219, (CEREMADE / INRIA, Paris, France), pp. 15-30, Springer, London, June 26-28 1996.
[33] . M. ter Haar Romeny, "Applications of scale-space theory," in Gaussian Scale-Space Theory (J. Sporring, M. Nielsen, L. Florack, and P. Johansen, eds.), Computational Imaging and Vision, pp. 3-19, Dordrecht: Kluwer Academic Publishers, 1997.
[34] B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, eds., "Scale-Space '97: Proc. First Internat. Conf. on Scale-Space Theory in Computer Vision", vol. 1252 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 1997.
[35] B. M. ter Haar Romeny: "Front-End Vision and Multiscale Image Analysis. An Interactive Tutorial". Kluwer Academic Publishers, Dordrecht, the Netherlands, 2000.
Romeny-c.nb 33
[36] A. H. Salden, B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, "A complete and irreducible set of local orthogonally invariant features of 2-dimensional images," in Proceedings 11th IAPR Internat. Conf. on Pattern Recognition (I. T. Young, ed.), (The Hague, the Netherlands), pp. 180-184, IEEE Computer Society Press, Los Alamitos, August 30-September 3 1992.
[37] A. H. Salden, B. M. ter Haar Romeny, and M. A. Viergever, "Linear scale space theory from physical properties," J. of Mathematical Imaging and Vision, vol. 9. no.2, pp. 103-140, 1998.
[38] O. Scherzer, J. Weickert, "Relations between regularization and diffusion filtering", J. Math. Imagind and. Vision, in press, 2000.
[39] L. Schwartz, "Théorie des Distributions", Hermann, Paris, 1951, 2nd edition 1966.
[40] J. Sporring, M. Nielsen, L. Florack: "Gaussian Scale-Space Theory", Kluwer Academic Publishers, Dordrecht, the Netherlands, 1997.
[41] J. Weickert, S. Ishikawa, A. Imiya, "On the history of Gaussian scale-space axiomatics", in J. Sporring, M. Nielsen, L. Florack, P. Johansen (Eds.), Gaussian scale-space theory, Kluwer, Dordrecht, 45-59, 1997.
[42] J. Weickert, "Anisotropic diffusion in image processing," ECMI Series, Teubner, Stuttgart, 1998.
[43] J. Weickert, S. Ishikawa, A. Imiya, "Linear scale-space has first been proposed in Japan", J. Math. Imag. Vision, Vol. 10, 237-252, 1999.