Some words on Stochastic Eigen-Analysisweb.mit.edu/sea06/agenda/talks/Edelman.pdf · Chikuse (Statistics on manifolds) ... Stochastic Differential Operators. Stochastic Operators

Some words on Stochastic Eigen-Analysis

Alan Edelman Raj Rao

Dept of MathematicsComputer Science & AI Laboratories

MITJuly 10, 2006

Some words on Stochastic Eigen-Analysis

Alan Edelman Raj Rao

Dept of MathematicsComputer Science & AI Laboratories

MITJuly 10, 2006

Just when you thought mathematics just about

wrapped up …

1. Orthogonal Polynomials & Special Functions

2. Convolutions3. Stochastic Differential Operators

Just when you thought mathematics just about

wrapped up …

1. Orthogonal Polynomials & Special Functions

2. Convolutions3. Stochastic Differential Operators

Pre & Early Computer Days

The Bateman Manuscript

Project

The web era

The SEA era

Ahead of its time Orthogonal Polynomials & Random Matrices:

A Riemann-Hilbert Approach MOPS: Dumitriu

Koev

Anshelevich (Free Meixner poly.)

Chikuse (Statistics on manifolds)

Just when you thought mathematics just about

wrapped up …

1. Orthogonal Polynomials & Special Functions

2. Convolutions3. Stochastic Differential Operators

Classical Convolutions

Plus

-3 -2 -1 0 1 2 30

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

Prob

abili

ty

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

Pro

babi

lity

Y=randn(n,2n)B=Y*Y’

zm2+(2z-1)m+2=0

+

X =randn(n,n)A=X+X’

m2+zm+1=0

-2 -1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

Pro

babi

lity

A+Bm3+(z+2)m2+(2z-1)m+2=0

Times

-3 -2 -1 0 1 2 30

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

Prob

abili

ty

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

Pro

babi

lity

X =randn(n,n)A=X+X’

m2+zm+1=0

Y=randn(n,2n)B=Y*Y’

zm2+(2z-1)m+2=0

*

-2 -1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

Pro

babi

lity

A*Bm4z2-2m3z+m2+4mz+4=0

-3 -2 -1 0 1 2 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

Pro

babi

lity

The convolutions (Free Prob)

Spectrum of Sample Covariance Matrix

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 40

0.2

0.4

0.6

0.8

1

1.2

1.4

x

Pro

babi

lity

c = 0.5c = 0.1

- Convolution is highly non-linear

- Density is function of Sensors/Snapshots, eig(R)

- Symbolic package (RMTool) to compute density

. Moments (canonically) in closed form!

1 3

0.4

0.6

eig(R) Marcenko-Pastur SCM spectrum

0 1 2 3 4 5 6 7 80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x

Pro

babi

lity

c = 0.5c = 0.1

Eigenvalues of true covariance matrix

Eigenvalues blurred (free convolution)

R^ = R1/2 W(c) R1/2

Two distinct subspaces

Blurring of eigenvaluesbecause of insufficient sample support

Free “Deconvolving” of a singleSample Covariance Matrix

There is no structure visible to the eye, but the subspace structure can be deduced by free deconvolution

Eigenvalues blur because of limited data

“Convolution” <-> “Deconvolution”

• Model based– moment matching + “second” order freeness

(with Speicher + Mingo)– parametric

• Non-model based– Stieltjes transform-to-resolvent matching– Connection to Lanczos, GMRES– (with Per-Olof Persson)– non-parametric

Free probability in SEA’06

• Speicher (Survey)• Chatterjee (Rate of convergence)• Speicher + Mingo (Fluctuations & 2nd order

freeness)• Anshelevich (Free Meixner polynomials)• Demni (Processes)• Burda (Free Levy matrices)• Kargin (Large deviations)• Rashidi Far (Operator values free probability)

Just when you thought mathematics just about

wrapped up …

1. Orthogonal Polynomials & Special Functions

2. Convolutions3. Stochastic Differential Operators

Stochastic Operators

• Ito & Stratonovich• Many recent methods• Whole Field of stochastic differential

equations–MATLAB SPEAK:

• rand + “\” well studied• rand + “eig” missing

Stochastic Operator Limit

,

N(0,2)χχN(0,2)χ

χN(0,2)χχN(0,2)

nβ21~H

β

β2β

2)β(n1)β(n

1)β(n

βn

⎟⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜⎜

⎝

⎛

−−

−

,dWβ

2x dxd

2

2

+−

,Gβ

2HH nnβn +≈ ∞

… … …

21

Those betas• Real Numbers: x β=1• Complex Numbers: x+iy β=2• Quaternions: x+iy+jz+kw β=4• β=2½? x+iy+jz

Other (Math) Talks in SEA’06

• Appearance of “universal” distributions– Kuijlaars, Baik, Johnstone, El Karoui, Dieng,

Sutton, Rider, Sasamoto, Seba

• Causal sets, airplane boarding– Bachmat

• Complex systems– Ergün, Sethna, Timme

• Principal component analysis– Paul, Onatski

• Applications & more!