Signal Processing and Representation Theory Lecture 1.

Signal Processingand

Representation Theory

Lecture 1

Outline:• Algebra Review

– Numbers– Groups– Vector Spaces– Inner Product Spaces– Orthogonal / Unitary Operators

• Representation Theory

Algebra Review

Numbers (Reals)Real numbers, ℝ, are the set of numbers that we express in decimal notation, possibly with infinite, non-repeating, precision.

Algebra ReviewNumbers (Reals)Example: =3.141592653589793238462643383279502884197…

Completeness: If a sequence of real numbers gets progressively “tighter” then it must converge to a real number.

Size: The size of a real number aℝ is the square root of its square norm: 2aa

Algebra Review

Numbers (Complexes)Complex numbers, , are the set of numbers that we ℂexpress as a+ib, where a,b and ℝ i= .

Example: ei=cos+isin

1

Algebra Review

Numbers (Complexes)Let p(x)=xn+an-1xn-1+…+a1x1+a0 be a polynomial with ai .ℂ

Algebraic Closure:

p(x) must have a root, x0 in :ℂ

p(x0)=0.

Algebra Review

Numbers (Complexes)Conjugate: The conjugate of a complex number a+ib is:

Size: The size of a real number a+ibℂ is the square root of its square norm:

ibaiba

22)()( baibaibaiba

Algebra ReviewGroupsA group G is a set with a composition rule + that takes two elements of the set and returns another element, satisfying:

– Asscociativity: (a+b)+c=a+(b+c) for all a,b,cG.– Identity: There exists an identity element 0G such

that 0+a=a+0=a for all aG.– Inverse: For every aG there exists an element -aG

such that a+(-a)=0.

If the group satisfies a+b=b+a for all a,bG, then the group is called commutative or abelian.

Algebra ReviewGroupsExamples:

– The integers, under addition, are a commutative group.– The positive real numbers, under multiplication, are a

commutative group.– The set of complex numbers without 0, under

multiplication, are a commutative group.– Real/complex invertible matrices, under multiplication

are a non-commutative group.– The rotation matrices, under multiplication, are a non-

commutative group. (Except in 2D when they are commutative)

Algebra Review

(Real) Vector SpacesA real vector space is a set of objects that can be added together and scaled by real numbers.

Formally:

A real vector space V is a commutative group with a scaling operator:

(a,v)→av,

a , ℝ vV, such that:

1. 1v=v for all vV.

2. a(v+w)=av+aw for all a , ℝ v,wV.

3. (a+b)v=av+bv for all a,b , ℝ vV.

4. (ab)v=a(bv) for all a,b , ℝ vV.

Algebra Review

(Real) Vector SpacesExamples:• The set of n-dimensional arrays with real coefficients is a

vector space.

• The set of mxn matrices with real entries is a vector space.

• The sets of real-valued functions defined in 1D, 2D, 3D,… are all vector spaces.

• The sets of real-valued functions defined on the circle, disk, sphere, ball,… are all vector spaces.

• Etc.

Algebra Review

(Complex) Vector SpacesA complex vector space is a set of objects that can be added together and scaled by complex numbers.

Formally:

A complex vector space V is a commutative group with a scaling operator:

(a,v)→av,

a , ℂ vV, such that:1. 1v=v for all vV.

2. a(v+w)=av+aw for all a , ℂ v,wV.

3. (a+b)v=av+bv for all a,b , ℂ vV.

4. (ab)v=a(bv) for all a,b , ℂ vV.

Algebra Review(Complex) Vector SpacesExamples:• The set of n-dimensional arrays with complex coefficients

is a vector space.• The set of mxn matrices with complex entries is a vector

space.• The sets of complex-valued functions defined in 1D, 2D,

3D,… are all vector spaces.• The sets of complex-valued functions defined on the

circle, disk, sphere, ball,… are all vector spaces.• Etc.

Algebra Review

(Real) Inner Product SpacesA real inner product space is a real vector space V with a mapping V,V→ℝ that takes a pair of vectors and returns a real number, satisfying:

u,v+w= u,v+ u,w for all u,v,wV. αu,v=αu,v for all u,vV and all αℝ. u,v= v,u for all u,vV. v,v0 for all vV, and v,v=0 if and only if v=0.

Algebra Review(Real) Inner Product SpacesExamples:

– The space of n-dimensional arrays with real coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then:

v,w=v1w1+…+vnwn

– If M is a symmetric matrix (M=Mt) whose eigen-values are all positive, then the space of n-dimensional arrays with real coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then:

v,wM=vMwt

Algebra Review

(Real) Inner Product SpacesExamples:

– The space of mxn matrices with real coefficients is an inner product space.If M and N are two mxn matrices then:

M,N=Trace(MtN)

Algebra Review

(Real) Inner Product SpacesExamples:

– The spaces of real-valued functions defined in 1D, 2D, 3D,… are real inner product space.If f and g are two functions in 1D, then:

– The spaces of real-valued functions defined on the circle, disk, sphere, ball,… are real inner product spaces.If f and g are two functions defined on the circle, then:

dxxgxfgf )()(,

2

0)()(, dgfgf

Algebra Review

(Complex) Inner Product SpacesA complex inner product space is a complex vector space V with a mapping V,V→ℂ that takes a pair of vectors and returns a complex number, satisfying:

u,v+w= u,v+ u,w for all u,v,wV. αu,v=αu,v for all u,vV and all αℝ.

– for all u,vV. v,v0 for all vV, and v,v=0 if and only if v=0.

uv,vu,

Algebra Review(Complex) Inner Product SpacesExamples:

– The space of n-dimensional arrays with complex coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then:

– If M is a conjugate symmetric matrix ( ) whose eigen-values are all positive, then the space of n-dimensional arrays with complex coefficients is an inner product space.If v=(v1,…,vn) and w=(w1,…,wn) then:

v,wM=vMwt

nn11 wv...wvwv, tMM

Algebra Review

(Complex) Inner Product SpacesExamples:

– The space of mxn matrices with real coefficients is an inner product space.If M and N are two mxn matrices then:

NMNM, tTrace

Algebra Review

(Complex) Inner Product SpacesExamples:

– The spaces of complex-valued functions defined in 1D, 2D, 3D,… are real inner product space.If f and g are two functions in 1D, then:

– The spaces of real-valued functions defined on the circle, disk, sphere, ball,… are real inner product spaces.If f and g are two functions defined on the circle, then:

dxxgxfgf )()(,

2

0)()(, dgfgf

Algebra Review

Inner Product SpacesIf V1,V2V, then V is the direct sum of subspaces V1, V2, written V=V1V2, if:

– Every vector vV can be written uniquely as:

for some vectors v1V1 and v2V2.21 vvv

Algebra Review

Inner Product SpacesExample:

If V is the vector space of 4-dimensional arrays, then V is the direct sum of the vector spaces V1,V2V where:

– V1=(x1,x2,0,0)

– V2=(0,0,x3,x4)

Algebra Review

Orthogonal / Unitary OperatorsIf V is a real / complex inner product space, then a linear map A:V→V is orthogonal / unitary if it preserves the inner product:

v,w= Av,Awfor all v,wV.

Algebra Review

Orthogonal / Unitary OperatorsExamples:

– If V is the space of real, two-dimensional, vectors and A is any rotation or reflection, then A is orthogonal.

A=

v2v1

A(v2)

A(v1)

Algebra Review


– If V is the space of real, three-dimensional, vectors and A is any rotation or reflection, then A is orthogonal.

A=

Algebra Review


– If V is the space of functions defined in 1D and A is any translation, then A is orthogonal.

A=

Algebra Review


– If V is the space of functions defined on a circle and A is any rotation or reflection, then A is orthogonal.

A=

Algebra Review


– If V is the space of functions defined on a sphere and A is any rotation or reflection, then A is orthogonal.

A=

Outline:• Algebra Review

• Representation Theory– Orthogonal / Unitary Representations– Irreducible Representations– Why Do We Care?


Orthogonal / Unitary RepresentationAn orthogonal / unitary representation of a group G onto an inner product space V is a map that sends every element of G to an orthogonal / unitary transformation, subject to the conditions:

1. (0)v=v, for all vV, where 0 is the identity element.

2. (gh)v=(g) (h)v


Orthogonal / Unitary RepresentationExamples:

– If G is any group and V is any vector space, then:

is an orthogonal / unitary representation.

– If G is the group of rotations and reflections and V is any vector space, then:


vvg )(

vgvg )det()(



– If G is the group of nxn orthogonal / unitary matrices, and V is the space of n-dimensional arrays, then:


vgvg )(



– If G is the group of 2x2 rotation matrices, and V is the vector space of 4-dimensional real / complex arrays, then:


),(),,(,,,)( 43214321 xxgxxgxxxxg


Irreducible RepresentationsA representation , of a group G onto a vector space V is irreducible if cannot be broken up into smaller representation spaces.

That is, if there exist WV such that:

(G)WW

Then either W=V or W=.

Representation TheoryIrreducible RepresentationsIf WV is a sub-representation of G, and W is the space of vectors perpendicular to W:

v,w=0for all vW and wW, then V=WW and W is also a sub-representation of V.For any gG, vW, and wW, we have:

So if a representation is reducible, it can be broken up into the direct sum of two sub-representations.

wvgwggvgwgv ,,,0 11


Irreducible RepresentationsExamples:

– If G is any group and V is any vector space with dimension larger than one, then:

is not an irreducible representation.

vvg )(


Irreducible RepresentationsExamples:

– If G is the group of 2x2 rotation matrices, and V is the vector space of 4-dimensional real / complex arrays, then:

is not an irreducible representation since it maps the space W=(x1,x2,0,0) back into itself.

),(),,(,,,)( 43214321 xxgxxgxxxxg


Why do we care?


Why we careIn shape matching we have to deal with the fact that rotations do not change the shape of a model.

=


Exhaustive SearchIf vM is a spherical function representing model M and vn is a spherical function representing model N, we want to find the minimum over all rotations T of the equation:

NMNM

NMNM

vTvvv

vTvvvT

,2

),,(22

2

D


Exhaustive SearchIf V is the space of spherical functions then we can consider the representation of the group of rotations on this space.

By decomposing V into a direct sum of its irreducible representations, we get a better framework for finding the best rotation.


Exhaustive Search (Brute Force)Suppose that {v1,…,vn} is some orthogonal basis for V, then we can express the shape descriptors in terms of this basis:

vM=a1v1+…+anvn

vN=b1v1+…+bnvn


Exhaustive Search (Brute Force)Then the dot-product of M and N at a rotation T is equal to:

n

jijiji

n

jjj

n

iii

n

jjj

n

iiiNM

vTvba

vTbva

vbTvavTv

1,

11

11

,

,

,,

n

jijijiNM vTvbavTv

1,

,,


Exhaustive Search (Brute Force)So that the nxn cross-multiplications are needed:

T(vn)

vM

v1

v2

vn

=

+

+

+

T(v1)

=

+

+

+

T(v2)

T(vN)… …


Exhaustive Search (w/ Rep. Theory)Now suppose that we can decompose V into a collection of one-dimensional representations.

That is, there exists an orthogonal basis {w1,…,wn} of functions such that T(wi)wiℂ for all rotations T and hence:

wi,T(wj)=0 for all i≠j.


Exhaustive Search (w/ Rep. Theory)Then we can express the shape descriptors in terms of this basis:

vM=α1w1+…+αnwn

vN=β1w1+…+βnwn


Exhaustive Search (w/ Rep. Theory)And the dot-product of M and N at a rotation T is equal to:

n

iiiii

n

jijiji

n

jjj

n

iii

n

jjj

n

iiiNM

wTw

wTw

wTw

wTwvTv

1

1,

11

11

,

,

,

,,

n

iiiiiNM wTwvTv

1

,,


Exhaustive Search (w/ Rep. Theory)So that only n multiplications are needed:

T(wn)

vM

w1

w2

wn

=

+

+

+

T(w1)

=

+

+

+

T(w2)

T(vN)… …

Signal Processing and Representation Theory Lecture 1.

Documents

real vector space v

set of numbers

complex vector space

set of complex numbers

commutative slide

sequence of real numbers

positive real numbers

sets of real